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Abstract 



We discuss research done in two important areas of nonequilibrium statistical 
mechanics: fluctuation dissipation relations and dynamical fluctuations. The work 
discussed here was reported before in [2J |3J [H [701 [7T] . 

In equilibrium systems the fluctuation-dissipation theorem gives a simple relation 
between the response of observables to a perturation and correlation functions in 
the unperturbed system. Our contribution here is an investigation of the form of 
the response function for systems out of equilibrium. We found that the response 
function can generally be written as the sum of two correlation functions. One 
correlation function is linked to entropy exchange with the environment, and thus 
to heat dissipation. The other correlation function has to do with a quantity which 
we call traffic and which describes in a sense the activity of the system. The results 
are applied to several explicit examples for which simulations have provided some 
visualization. 

Furthermore, we use the theory of large deviations to examine dynamical 
fluctuations in systems out of equilibrium. In dynamical fluctuation theory we 
consider two kinds of observables: occupations (describing the fraction of time 
the system spends in each configuration) and currents (describing the changes of 
configuration the system makes). We explain how to compute the rate functions of 
the large deviations, and what the physical quantities arc that govern their form. 
As for fluctuation-dissipation relations, entropy and traffic are the main ingredients. 
Moreover, the rate function that governs the joint probabilities of occupations and 
currents is explicitly computed for the classes of models considered and is expressed 
in terms of entropy and traffic. The rate function for the occupations can be 
expressed entirely in terms of traffic. We also show that this traffic can be seen 
as a thermodynamic potential for currents. Finally, for the close-to-equilibrium 
regime, known variational principles as the minimum entropy production principle 
are recovered. 
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k B 


Boltzmann's constant, usually we work in units in which ks = 1 




inverse temperature, 


U,V 


potentials, 


F,f,g 


forces, 


W,Q 


work and heat, 


n 


chemical potential. 



Configurations and trajectories 

f2 configuration/state space, 

x, y, . . . configurations/states, 
t,s,T times, 
Xt configuration at time t, 

A t protocol, 

u! = (x t )o<t<T trajectory/path during an interval [0,T], 

7r kinematical time-reversal operator 

changing the signs of velocities, 
t)o<t<T time-reversal of trajectories, 

path- probability measure for paths given Xq, 
probability distribution of initial configuration, 
path-probability measure with 
initial state sampled from /io, 
path-probability measure with 
reversed protocol, 
: / dV Xo (w)f(uj) expectation value of a function /, 
= J cfP^ (w)/(u;) expectation value of a function /, 

Radon-Nikodym derivative, 

dV* 

log (oj) the action, 

time-evolved probability distribution, 
stationary distribution, 
probability current when in /j. 



9ui = (ttxt- 
Mo 



dV R 

(/H), 

(/(")>*, 

s&<«> 

A(w) = - 

p(x) 
3n( x ) 
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Entropy and traffic 

S(u>) = log fpu a e (w) entropy flux into environment, 

Sfj_(uj) = log av^e ( a; ) measure of irreversibility, 

s(/i) Shannon/ Gibbs entropy of /x, 

cr(/i) expected entropy production rate when in /x, 

S ex (ui) = A r (9oj) — A(uj) excess entropy flux, 

T ex (u) = A r (9cj) + A(uj) excess traffic, 

r(/Lt) expected traffic rate when in \x. 



Markov jump processes 

x, y, . . . configurations, 

kt{x,y) transition rates, 

^t(x) = J2 y h{x, y) escape rate. 



Diffusions 



X 


position, 




components of the position 


V 


velocity, 


m 


mass, 


7 


friction coefficient, 


X 


mobility, 


D 


diffusion coefficient, 


B, 


Wiener process. 



Fluctuation-dissipation 

h t time-dependent amplitude of the perturbation, 

V perturbing potential, 

Q observable, not to be confused with heat, 

RQv{t, s) response function, 

t(u;,s) functional derivative of excess traffic w.r.t. h s . 



Dynamical fluctuations 

Puj empirical occupation vector/density, 

J w empirical current, 

[i fluctuation of the occupations, 

j fluctuation of the current, 

/ rate function or fluctuation functional, 

Kf a quantity K computed in a dynamics determined by a force /, 

/zi small deviation of fi from p, 

ji small deviation of j from j p . 
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Chapter 1 

Introduction 



In this chapter we quickly review some important concepts in thermodynamics 
and statistical mechanics that are relevant for the rest of this thesis. For a more 
thorough introduction we refer to standard textbooks. After this we introduce 
the reader to the realm of nonequilibrium phenomena and motivate the research 
discussed in this thesis. 




1.1 Thermodynamics 



Thermodynamics studies energy conversions between 
mechanical work and heat. Although several results 
in thermodynamics date back to the seventeenth 
century, the theory as we now know it had its 
major breakthrough in the nineteenth century, starting 
with groundbreaking theoretical considerations by Sadi 
Carnot. It is no coincidence that this happened in the 
century of the industrial revolution. Thermodynamics 
provided a deep and indispensable understanding of the 
principles by which engines (and refrigerators) operate, 
and the fundamental limits they must obey. It is 
also a logical starting point for our discussion. In 
the following pages we quickly review some aspects of 
thermodynamics that are relevant to the rest of this 
thesis; it is certainly not our goal to review the whole 
theory of thermodynamics. 



Figure 1.1: Sadi Carnot, 
1796 - 1832. 
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Classical thermodynamics as developed in the nineteenth century is a theory of 
macroscopic systems. As we now know, a macroscopic system consists of a large 
(10 23 ) number of particles (atoms/molecules). Thermodynamics thus describes 
systems with variables such as temperature, pressure, volume, etc, ignoring the 
microscopic details of the system (on the level of the molecules). When these 
variables change in time we speak of a thermodynamic process. 

Equilibrium As this text is situated in nonequilibrium thermal physics, it is useful 
to describe what it means to be in equilibrium. With equilibrium we mean the 
following: 

• Two systems are said to be in mechanical equilibrium with each other when 
the pressures (P) they exert on each other are equal. If the pressures were 
different, one system would do work on the other, causing a change in the 
volumes (V). 

• Two systems that can exchange particles are in diffusive equilibrium when 
their chemical potentials (/i) are the same. If they are not then there will be 
a net current of particles from one system to another, causing the particle 
numbers (TV) of the system to change. 

• Two systems are in thermal equilibrium when, after being brought into 
thermal contact with each other, they do not exchange heat. In this case 
their temperatures (T) are the same. If the temperatures are not the same, 
there will be heat flow from one system to the other, causing a change in a 
new quantity, named entropy (S). (We will come back to this later). 

• Two systems are in thermodynamic equilibrium if they arc in mechanical, 
diffusive and thermal equilibrium. 

Usually we do not speak about two systems but about one system and its 
environment, and say that a system is in equilibrium if it is in equilibrium with 
its environment. 

These definitions already tell us that we distinguish three ways of exchanging 
energy between systems: work, particle exchange and heat exchange. Furthermore 
six variables are introduced, grouped in three pairs: P, V and N and T, S. These 
variables are not independent. Simple systems, such as gases (or more generally 
pure fluids) can be described by taking one variable of each group, depending on 
the interaction of the system with its environment. This collection of variables 
is then called the (equilibrium) state of the system. For example, if the system 
is mechanically isolated from the environment, then its volume is fixed and can 
be used to describe the system. One can also fix the pressure and let the volume 
vary, thus allowing exchange of work, and so on. For example a system that can 
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exchange work and heat but not particles with its environment is best described 
by the variables P,N,T. 

Note that V, N, S are extensive variables, i.e. they scale with the 'size' of the 
system (e.g. if we take two copies of the same system, it has twice the volume, 
twice the number of particles and twice the entropy). In contrast P,fj.,T are 
intensive variables, i.e. they are independent of the size of the system. 

There is a special class of thermodynamic processes which have the following 
property: if the process is run backwards, eventually the system and its 
environment return to the same equilibrium state they had before the original 
process. We call this a reversible process. Actually, for this to be true in real 
systems, the process should go infinitely slowly, and can in that case be described 
by a sequence of equilibrium states. As it turns out, many processes in real life 
are slow enough for this description to be a reasonable approximation. 

Of course, not all quantities mentioned here have a clear intuitive meaning. 
Especially the quantities heat and entropy that have to do with thermal 
equilibrium are vague. The laws of thermodynamics give a further specification 
of these concepts. We discuss them here shortly, as the understanding of entropy 
and its role in noncquilibrium systems is very important throughout this thesis. 

The first law The internal energy U of a system corresponds microscopically to 
the sum of all kinetic energies of the particles and all their interaction potentials. 
The first law of thermodynamics then dictates that a small change of the energy 
dU of a system can only be caused by work, heat or a particle flow: 

dU =dQ+dW + /j,dN 

where dQ is an infinitesimal amount of heat flow into the system and dW is an 
infinitesimal amount of work done on the system. The notation with d stems from 
the fact that heat and work can't generally be expressed as the difference of a state 
function (a function only depending on the state of the system, not on its history). 
If the system contains different species of particles with chemical potentials y,i 
then the last term should be replaced by HidNi. 

For a system with a fixed number of particles and a fixed volume, the first law tells 
us that the energy of a system can only change through heat flow. In a sense this 
is a definition of heat: it is an energy transfer that is not work and not a particle 
current. 

To have a better understanding of the concept of heat let us make a little thought 
experiment. Think of a box containing a gas (system). We imagine the box 
standing in a room filled with air (environment). The box is closed and has a 
fixed volume. Going down to the microscopic level (leaving thermodynamics for 
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Figure 1.2: The experiment with which J. P. Joule showed that heat and mechanical 
work are both forms of energy transfer. 

a moment), we can imagine that particles outside the box collide with the box, 
thus exchanging work with the box. The particles in the box also exchange work 
with the box. In this way energy can be transferred between the system and the 
environment. However when we zoom out to the macroscopic level, we can no 
longer see the individual collisions of all the particles, and if we do not see forces, 
we can't compute their work. We only see that energy is exchanged. In this way 
heat is defined as energy exchange due to work of forces we can't see from our 
macroscopic point of view. More generally heat can also be particle exchange of 
particles we can't see. Note that this definition is arbitrary in that it depends on 
the level of description of the system. 

The second law This law is used to define the quantity entropy. It states that 
there exists a state function, called the entropy S, such that 

AS>/f 

where AS = Sf — Si is the change in entropy of the system from the initial value Si 
to the final value Sf, and the integral is over a thermodynamic process. Moreover, 
only for a reversible process the inequality becomes an equality, thereby exactly 
defining entropy changes. Apart from defining the entropy, this second law is also 
a restriction on the heat flow during a process. For example this law predicts that 
heat never spontaneously flows from a cold to a hot object. It also places a bound 
on engines that extract work by utilizing two reservoirs at different temperatures. 

For an isolated system (i.e. dQ = 0), what does it mean to be 'in equilibrium'? 
From the second law we see that any process the system undergoes will increase 
its entropy (or leave it unchanged) . Only when the system reaches its equilibrium 
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state, the entropy does not change. One can thus characterize equilibrium for an 
isolated system as the state with maximum entropy. 



Thermodynamic potentials Thermodynamic potentials are (scalar) functions 
that describe the thermodynamic state of a system. From them, many relations 
between thermodynamic quantities can be derived. Therefore they play a role in 
thermodynamics comparable to the Lagrangian or Hamiltonian in mechanics. Part 
of this thesis discusses the use of thermodynamical potentials in noncquilibrium 
systems. Because of this, we provide here a quick introduction to these potentials 
in thermodynamics. 

The first and most intuitive thermodynamic potential is the internal energy of the 
system: U . We can use the first law for a reversible process to write 

dU = TdS - PdV + fjdN 

where PdV is the work done by the system on its environment. From this we can 
see that the natural variables of U are S, V and N, while the variables T, P, /j, all 
depend on them through 

dU = dU dU = 

OS ' dV ' ON ^ 

The internal energy is thus fixed in a system for which S, V, N arc fixed and 
is thus the most natural potential here. Equivalently we can write the entropy 
S = S(U, V, N) which is most natural in a system with U, V, N fixed, i.e. a totally 
isolated system. As we already argued, the entropy characterizes the equilibrium 
state of such a system because it is maximal then. Such a characterization of 
equilibrium is a key feature of thermodynamic potentials. 

For a system with a fixed volume and particle number, but which can exchange 
heat with an environment which is in equilibrium at a fixed temperature, the 
natural variables are T,V,N. What characterizes equilibrium for such a system? 
To see this, we assume that the total of system plus environment is isolated. The 
energy of an isolated system is constant. So dU + dU e = 0, where dU is the 
change of energy of the system, and dll e of the environment. From the first law 
of thermodynamics, we see that for the environment (which is at equilibrium): 
dS e = TpdU e — —TpdU. The system is in equilibrium with its environment if the 
total of the two is in equilibrium. In that case the total entropy is maximal: 



d{S + S e )=0, d 2 (S + S e )<0 
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Written in terms of the system, this gives: 
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d(S--) = 0, d 2 (S--)<0 

This defines a thermodynamic potential S — U/T, but a slightly different potential 
is more commonly used: the Helmholtz free energy F = U — TS. We see that it is 
minimal in equilibrium. This free energy for an equilibrium system is thus defined 
by 

F(T, V, N) = w£[U(S, V, N) - TS] 

which means that it is a Lcgcndrc transform of the internal energy U. 
Similarly the enthalpy 

H = U + PV 

is defined for a system that is thermally isolated but can change its volume with 
an environment at constant pressure. In equilibrium we have 

H(S, P, N) = in£[U(S, V, N) + PV] 

The Gibbs free energy 

G(T, P,N) = U + PV -TS 

is defined for a system in contact with an environment at a constant temperature 
and pressure. More potentials can be defined when particle exchange with the 
environment is allowed. 

The collection of thermodynamic potentials, all connected through Lcgcndrc trans- 
forms, form a very powerful and useful theoretical formalism for thermodynamics 
of equilibrium systems. 

1.2 Equilibrium statistical mechanics 

Statistical mechanics, also called statistical thermodynamics, is the theoretical 
framework that explains thermodynamics as a set of macroscopic properties of 
materials, starting from the microscopic properties of the individual atoms or 
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molecules. But it docs more: it gives a more accurate description because it 
also describes the fluctuations from the expected macroscopic behaviour. It uses 
statistics/probability theory to be able to study systems that consist of a large 
number (10 23 ) particles. Historically, it was founded in the second half of the 
nineteenth century mainly by Ludwig Boltzmann and James Clerk Maxwell, but 
also by Gibbs, Einstein and Planck. Their statistical mechanics is a theory of 
equilibrium systems. We give a quick overview of some relevant aspects: 



Micro and macro In statistical mechanics one makes the difference between 
microstates and macrostates. A microstate (or microscopic configuration) of a 
system contains all microscopic information about the system, such as the positions 
and velocities of all its particles. A macrostate (or macroscopic configuration) 
gives a description of the system in terms of a few macroscopic properties, such 
as temperature and volume. Many microstates may correspond to the same 
macrostate. 



Entropy One of the basic postulates of statistical 
mechanics is that for an isolated system, all its 
possible microstates are equivalent, i.e one is not 
more important or more probable than another. As 
a consequence, for an isolated system, macrostates 
that correspond to more microstates are more 
probable than macrostates that correspond to 
fewer microstates. This is closely connected to 
Boltzmann 's famous definition of entropy as a 
statistical quantity: given a set of macrostates M, 
the entropy S is proportional to the logarithm 
of W, where W is the number of microstates 
corresponding to M: 



k log W 



S = k B log W 



(1.1) 



where ks is Boltzmann 's constant. For continuous 
microscopic configuration spaces, W is the 'volume' 
of the set of microstates corresponding to the 
macrostate. Saying that a system in equilibrium 
is characterized by a maximal entropy is thus 
the same as saying that it is characterized by 
the macrostates that correspond to the most 
microstates, i.e. the most probable macrostates. 
One of the major achievements of statistical 
mechanics is that this entropy coincides with its 
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Figure 1.3: 
gravestone. 



Boltzmann 's 
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thermodynamic counterpart mentioned above. As a testament to the significance 
of this formula it was engraved on the tombstone of Boltzmann. 

Ensembles The macrostate M that a system is in at any moment depends on 
the microstate x of the system: M = M(x). Statistical mechanics explains the 
measured value of M from the viewpoint of the microstate x the system can be in. 
It does this by working with ensembles. An ensemble is a set of (a large number 
of) imagined copies of a system: one for each microstate the system could be 
in. One then has to choose a probability distribution p(x) on this set of possible 
microstates. The probability of observing a certain macrostate M is then the sum 
(or integral) of p(x) over all microstates that correspond to M. The statistical 
average of M can be computed by 



where the sum can be an integral, depending on the system. Within the theory of 
equilibrium ensembles the definition of entropy was provided by Gibbs: 



which is equivalent to the Boltzmann entropy for the microcanonical ensemble (see 
below). 

As macroscopic systems typically have 10 23 particles, and thus a large number 
of microstates, the law of large numbers from probability theory dictates that 
the measured value of M is practically always equal to the computed average 
(M). Therefore, by choosing the right probability distribution p, one can recover 
thermodynamics by considering the statistical averages of macrostates. The most 
commonly used probability distributions are: 

• The microcanonical ensemble is used for isolated systems, i.e. systems that 
have a fixed energy E. The set of possible microstates is thus restricted to 
microstates that have an energy E. Apart from that an equal probability is 
assigned to all those microstates. 

• The canonical ensemble is used for systems that can exchange energy only 
in the form of heat with an environment at a fixed temperature T. The 
canonical distribution is 



X 




X 



p(x) 



Z 
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where Z is a normalization factor, called the canonical partition function, 
[3 = -j-Kp and E(x) is the energy of the system in microstate x. 

• The grand canonical ensemble is used for systems that can exchange heat and 
particles with a reservoir at a fixed temperature T and chemical potential ji: 

where Z is a normalization factor, called the grand canonical partition 
function, and N(x) is the number of particles of the system in microstate x. 

As an example of how the link to thermodynamics is made: we explicitly substitute 
the distribution for the canonical ensemble in the Gibbs entropy: 

S = k B \ogZ+ { -^ 

The average energy (E) coincides by the law of large numbers to the measured 
energy U , so 

--logZ = U -TS = F 

P 

where F is the Hclmholtz free energy of the system. 



1.3 Out of equilibrium 

In the end, the purpose of statistical mechanics is twofold: on the one hand 
to describe the macroscopic world, and derive its physical behaviour, starting 
from the microscopic level. On the other hand, statistical mechanics makes 
more predictions than thermodynamics, because it also describes the deviations 
(fluctuations) from the average behaviour. 

Equilibrium statistical mechanics is in this sense very powerful, because it 
is seen that measurable quantities, using the law of large numbers, can be 
computed through averaging over equilibrium distributions of microstates. These 
are themselves expressed solely in terms of external constraints (temperature, 
chemical potential), and conserved quantities like energy and particle number. 
Going beyond thermodynamics, for example the fluctuation-dissipation theorem 
is a useful and well-known physical relation. (Part of this text discusses the 
generalization of this to nonequilibrium systems). 
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However, up to the moment that this text is written, there is no general 
paradigmatic theoretical framework that describes systems out of equilibrium, 
neither in thermodynamics nor in statistical mechanics, not even for stationary 
systems. This poses a problem, because many processes in real life can't be 
described as reversible processes, and many systems are not in equilibrium. 

How to be out of equilibrium In general we distinguish two ways for systems 
to be out of equilibrium. First, systems that are in the process of relaxing to 
equilibrium: when a parameter determined by the environment changes, like 
temperature or volume, the system always needs some time to relax to that new 
equilibrium state. For example, when a hot cup of coffee is placed in a room at 
room temperature, heat will start to flow from the coffee to the air of the room (we 
ignore heat exchange by radiation with the walls of the room). After some time 
the coffee and the air will relax to a new equilibrium state: the coffee cools down 
to room temperature. Before that time, however, the coffee is not in equilibrium 
with the room, and the process is not reversible. If it was reversible, then we would 
not be surprised to observe heat to flow spontaneously from the air to the coffee, 
heating it up again. Sometimes the external parameters change continuously and 
fast enough such that the system never has enough time to relax to equilibrium. 
Think for example of combustion engines. 

Secondly there are systems that arc driven from equilibrium by what we call 
thermodynamic forces. Here we distinguish three subgroups, corresponding to 
thermal, diffusive and mechanical exchanges of energy: 

• Systems in contact with parts of the environment at different temperatures. 
For example a wall of a house: on one side it is in contact with the warm air 
inside, and on the other side it is in contact with cold air. In such systems 
there are constantly heat currents. A part of the environment that is in 
thermal contact with the system, is called a heat bath. The thermodynamic 
force here is the temperature difference. 

• Systems in contact with parts of the environment at different chemical 
potentials. Think of a cell membrane with a bigger particle density on the 
inside than on the outside. In such systems one sees particle currents. A 
part of the environment that is in diffusive contact with the system is called 
a particle reservoir. The thermodynamic force here is the chemical potential 
difference. 

• Systems under the influence of mechanical nonconservative forces. A 
nonconservative force is a force which is not the derivative from a potential. 
Think of the pressure difference that makes water run out of a tap. 
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To simplify this second group of systems, one usually assumes that the parts of 
the environment in contact with the system are and stay in equilibrium during 
the process. For this one should assume that there is no interaction between the 
different parts of the environment, and that the interaction between system and 
environment is weak, i.e. changes in each part of the environment are small enough 
such that it relaxes fast enough to equilibrium for our description of the system. 
One also assumes that the heat baths and particle reservoirs are big, such that 
during the process their temperatures and densities do not change measurably. 
Systems under such conditions can often relax to a stationary state. This is a 
state in which the variables (macrostatcs) with which we describe the system do 
not change in time. Such variables include heat or particle currents in contrast 
to equilibrium systems. One of the goals of noncquilibrium thermodynamics is to 
describe such stationary regimes and correctly predict the directions and sizes of 
their currents. 

Dynamics An important property of noncquilibrium systems, in contrast to 
equilibrium, is that time plays an important role. First of all because 
noncquilibrium processes are irreversible: an arrow of time is introduced. The 
question of how irreversibility emerges when going from the reversible microscopic 
world to the macroscopic world was already discussed by Boltzmann himself. 

Apart from that, it seems clear that noncquilibrium systems arc by their very 
nature dynamical. This is because systems are either out of equilibrium because 
they are driven from equilibrium and are in some stationary regime in which there 
are particle or heat currents present, or because they are in the process of relaxing 
to equilibrium (or to a stationary state) . This means that not only the microstates 
themselves arc important, but also the way in which they change in time. The 
dynamics of the process should enter our (theoretical) description. 

Statistical mechanics of stochastic processes: the mesoscopic level Ideally, 
one would therefore like to answer physical questions about average behaviour and 
fluctuations, by starting from the microscopic Hamiltonian dynamics. However, 
this is often just impossible. This is not necessarily a disaster, because we 
expect (hope) that not every detailed aspect of that dynamics is relevant for 
the macroscopic world. Therefore most models used in noncquilibrium statistical 
mechanics are already reduced descriptions, meaning that they do not contain 
all information of the microscopic world. For example, one usually reduces the 
description of the environment of the system to some parameters as temperature or 
chemical potential. On the other hand, to be able to describe fluctuations from the 
typical macroscopic behaviour, a more detailed description than the macroscopic 
one is needed. We then say that we are working on the mesoscopic level. 
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One should thus think of a mcsoscopic model as describing a small system, for 
which the description is not detailed enough to be Hamiltonian, but is not big 
enough for the law of large numbers to apply. Fluctuations around the statistical 
averages are important. As a consequence a model describing a mesoscopic system 
can be a stochastic process. A stochastic process is therefore a very important 
tool in nonequilibrium statistical processes. An important part of the present day 
research is therefore committed to finding 'recipes' for defining stochastic processes 
that are physically relevant. One way to do this is via the local detailed balance 
assumption, which will be discussed in the next chapters, and is used throughout 
this text. 

Applications As said before, many processes in everyday life arc not reversible, 
and many quantities of interest are some form of current, which is not found in 
equilibrium systems. On the macroscopic scale we find important examples in 
ecology and meteorology. In meteorology, we see that the temperature at the 
poles is on average much lower than at the equator. That is responsible for major 
energy currents in the earth's weather system. On a smaller scale one is interested 
in predicting local weather, like temperature changes and air flows (wind), given 
observational data concerning high and low pressure areas etc. In ecology one is 
interested in the energetic and material (food) flows between different species of 
organisms. 

On a much smaller scale we sec that mesoscopic models arc not only useful as 
a step towards macroscopic theories, but also as a description of interesting but 
very small systems. In recent years the nonequilibrium world of small systems has 
become more accessible experimentally, opening and expanding research areas as 
biophysics and nanotcchnology. In biophysics a lot of attention nowadays is going 
to e.g. the dynamics of DNA and RNA, transport of ions through cell membranes, 
molecular motors, etc. In nanotcchnology the electrical current in extremely small 
devices or parts of devices is central. Finally, in everyday life there are many finite 
networks, like transport and communication networks, on which processes occur 
which arc well described on the mesoscopic scale. 

A lot of interesting and useful research has been done and is being done in these 
areas, where explicit models (stochastic processes) are examined to derive results 
for each specific area of interest. However, this is not the goal of the research 
reported in this thesis. Instead, we try to find general physical structures of 
nonequilibrium statistical mechanics. Such a scheme should in the long run 
and together with many other contributions lead to a better understanding of 
nonequilibrium physics. Such an understanding should then give useful predictions 
applicable to the specific areas of interest mentioned above. 
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1.4 Outline and preview of the results 

Part I: Stochastic processes Because of the lack of a general theory, it may come 
as no surprise that the present day research into noncquilibrium systems focuses 
on simple physical systems. It is then important to define a stochastic process that 
models it correctly. The first part of this thesis therefore contains an introduction 
to the stochastic models that are used in the rest of the text, focusing on the 
relevant aspects for understanding the next two parts. 

Throughout this text, we restrict ourselves to classical systems, meaning that we 
do not consider relativistic or quantum effects. (Quantum mechanics only enters 
in some specific models in the discretization of configuration space) . 

Part II: Fluctuation-dissipation relations In this part of the thesis, we investigate 
how a system responds to a perturbation, namely a small change in its energy The 
central object that summarizes this is the response function. To be more precise, 
we denote the energy of the system in configuration x by U(x). As a perturbation 
this energy is changed by the addition of a potential: U — > U — h t V, where the 
time-dependent function ht is the amplitude of the perturbation. We restrict our 
possible class of systems to those in which a small amplitude h t only has small 
effects (excluding for example the regime of phase transitions). Then we can write 
the expectation of an observable Q in the perturbed system as the expectation in 
the unperturbed system plus a small correction: 



This defines the response function Rgyit, s). In equilibrium systems it is known 
that this response function can be written in terms of a correlation function in the 
unperturbed system (a system in equilibrium at inverse temperature /3): 



This is called the fluctuation-dissipation theorem. Our contribution has been to 
discuss the form of the response function for systems out of equilibrium. We 
found that the response function can generally be written as the sum of two 
correlation functions. One correlation function is linked to heat dissipation into 
the environment, and thus to entropy changes. The other correlation function has 
to do with a quantity which we call traffic, which describes in a sense the activity 
of the system. The main results of this part are summarized by formulae ()5.6|) . 
()5.14p and ()6.7[) . These results arc then applied to several explicit examples for 
which simulations have provided some visualization and verification. 




R QV (t,s)=p—(Q(x t )V(x a )) 



o 
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The research discussed in this part was reported before in [2J |3J H] . 

Part III: Dynamical fluctuations In this part of the thesis we use the theory 
of large deviations to examine certain fluctuations in systems out of equilibrium. 
For equilibrium statistical mechanics, large deviation theory provides a natural 
mathematical framework especially for the thermodynamic potentials discussed 
above. It is therefore useful to investigate that theory also for noncquilibrium 
systems. 

In this theory we consider the empirical occupation density p (which describes the 
fraction of time the system spends in each configuration) and the empirical current 
J (describing the changes of configuration the system makes) as observables. The 
probability is then considered that these observables take on some given values 
p ?s /j, and J j. Such a probability often shows an exponential decay whenever 
\x and j do not correspond to the typical values of the observables: 

Prob(p «/j,J«j)kj e -Ti(^j) 

where the duration T of the process is very large, and I(^,j) is called the rate 
function. 

The questions in this part of the thesis are then: how to compute the rate function, 
and what are the physical quantities governing its form? As in the previous 
part, entropy and traffic are the main ingredients. Moreover, the rate function 
is explicitly computed for the classes of models considered and expressed 
in terms of entropy and traffic, see (|10.25p and (|11.10p . The rate function for the 
occupations can be expressed entirely in terms of traffic. We also show that this 
traffic can be seen as a thermodynamic potential for currents. Finally, for the close- 
to-equilibrium regime, known variational principles such as the minimum entropy 
production principle are recovered. 

The research discussed in this part was published before in [701 E] ■ 
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Stochastic processes 



15 



16 



'Observe what happens when sunbeams are admitted into a building and shed 
light on its shadowy places. You will sec a multitude of tiny particles mingling in a 
multitude of ways... their dancing is an actual indication of underlying movements 
of matter that are hidden from our sight... It originates with the atoms which move 
of themselves. Then those small compound bodies that are least removed from the 
impetus of the atoms are set in motion by the impact of their invisible blows and 
in turn cannon against slightly larger bodies. So the movement mounts up from 
the atoms and gradually emerges to the level of our senses, so that those bodies 
are in motion that we sec in sunbeams, moved by blows that remain invisible." 



Lucretius, De rerum natura, ca. 60 BC. 



Chapter 2 

Describing stochastic processes 



Stochastic processes are a useful tool to explore nonequilibrium statistical 
mechanics. The basic ingredient here is not the microstate but the trajectory, i.e. 
the sequence of states the system visits during the process. In this chapter we 
therefore discuss how to work with such trajectories in statistical physics. To lift the 
mathematical models to a more physical level, the local detailed balance assumption 
is made and explained. As we will see, some well-known general physical results 
are a direct consequence of this assumption. 

2.1 Stochastic processes 

Mathematically we describe a system by its configuration or state. This state 
contains all information of the system that is relevant for our description. Let 
us denote it by x, being an element of the configuration space f2. This x can 
be for example the vector of positions and momenta of a gas, or the set of spin- 
configurations of a magnet, or maybe just the position of one particle submerged 
in a fluid. Consequently, the set can be continuous or discrete, unbounded, 
compact or even finite. 

In the course of time the configuration of the system will change, i.e. the 
configuration depends on time: x = x t . How it changes, is determined by the 
dynamics of the process. In classical mechanics, the dynamics at the microscopic 
level is Hamiltonian and therefore deterministic. Indeed, when all positions and 
momenta of the particles of a gas in an isolated container are given at a time t, 
one can in principle determine the positions and momenta at any later time. 
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Unfortunately, this is not always possible. Most of the times the system under 
consideration is not isolated, but interacts with some environment of which the 
exact configuration is unknown. Moreover it is usually even impossible to measure 
the exact configuration of the system itself. Because of such factors, uncertainty 
enters, and must enter our mathematical description. This gives rise to stochastic 
processes, and the configuration xt of the system at each time becomes a stochastic 
variable X t . A stochastic process is then the sequence of these stochastic variables: 

{X t \0 <t<T} 

When the history of the system is given, i.e. xt for t G] — oo,T], the stochastic 
dynamics of the process gives a probability to find the system in a configuration 
y at a later time. 

Again one has to be more modest, as the complete history of a system is rarely 
known. A widely used and usually well- working approximation is the Markov 
approximation. This consists of the assumption that the probability of finding the 
system at a later time in some configuration, only depends on the configuration of 
the present, not the past. Informally, for a sequence of times ti < < ■ ■ ■ < t n < t: 

Prob{x t = x\x tn = y n , ...,x tl =yi) = Prob(x t = x\x tn = y n ) (2.1) 

A Markov process is a stochastic process for which the Markov assumption (|2.ip 
is valid. Note that Hamiltonian dynamics, governing the evolution of the state 
(x,v), although deterministic, also satisfy this assumption. In the rest of this text 
we always assume our processes to be Markovian. In many cases the dynamics of 
the Markov process have the property that 

Prob{xt = x\x s = y) = Prob(xt- s = x\x^ = y) Vs < t 

In such cases we speak of time-independent dynamics (or time-homogeneous 
dynamics). However, this is not always the case in this thesis. 

For the reader who is unfamiliar with probability theory and stochastic processes, 
we refer to |4b] for a thorough introduction. 

A particle in a box As an easy introductory example, consider a box with volume 
V filled with air. In this box we place one charged test-particle. We also apply an 
electric field to move the particle in our favourite directions. If we knew all the 
positions and velocities of all the particles in the box, in principle the trajectory 
of test-particle could be computed with Hamilton's equations. Practically, this 
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is not possible, but suppose that in some way the position of the test-particle 
can be measured at time intervals of 1 second, the first measurement being at 
time zero. The precision of the measurement is limited: one can only determine 
the position up to a small volume AV . Therefore we divide the box into N = 

Vj AV parts, labelled by x = 1,2, Given the configuration (position) x after 

n seconds, denoted by x n , the position x„+i is not determined exactly. There 
are usually several possible positions that the particle can be in, each having a 
probability determined by the temperature of the air, the electric and magnetic 
fields, etc. These probabilities can also depend on the previous positions of the 
particle. However, let us assume that the interaction with the air molecules is 
sufficiently strong compared to the influence of the electromagnetic fields and 
inertial effects, such that the particle loses its memory quickly enough for the 
Markov assumption to be valid. 

Having this physical example in our mind, let us write the probabilities of the 
successive positions (transition probabilities) as follows: 

P n (x, y) = Prob(x n+1 = y\x n = x) 

Mathematically, this defines what is called a Markov chain, which is the simplest 
example of a Markov process. In the next two chapters we discuss two other classes 
of Markov processes: Markov jump processes and diffusions, but in this chapter 
we use these simple Markov chains to illustrate the introduced concepts. 

The N x N matrix P n is called the transition matrix at time n, and it determines 
the dynamics of the process. Note that we should have that ^2 y P n (x,y) = 1 
for every n, where the sum is of course over all configurations. When the 
electromagnetic fields are constant in time, the dynamics are time-homogeneous: 
P n = P. When the fields are time-dependent, so are the dynamics. The time- 
dependence of the fields is determined by some parameter A t which continuously 
changes in the time t, so P n = P\ n . This A t is controlled externally, and we 
imagine it to be deterministic. It is called the protocol. 

2.2 Trajectories 

A realization of a stochastic process is called a trajectory or path. We denote it 
by uj = (xt)o<t<T- If the dynamics of the stochastic process are given, one can 
in principle compute the probability measure on such paths uj. Suppose that the 
configuration of the system at time zero xq is given. We denote by dP Xo (uj) the 
path-probability measure of uj given xq. With it we can compute the expectation 
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values of observables: take any observable f{uS) that depends on the trajectory. 
Then its expectation value is 

</M> ao = / <8>«oM/M ( 2 - 2 ) 

where one integrates (or sums, depending on the model) over all possible 
trajectories that start from xq at time zero. More generally, given a probability 
distribution (or density) fj,Q of initial configurations, we denote the probability 
measure of a path by d'P Jtl0 (w) = no(xo) dP Xo (oj) and the expectation value of /(w) 
by 

where integration/summation is now over all possible trajectories. 



Basic probabilistic rules dictate that the path-probability measure should be 
normalized: 

JdVM = l (2.3) 

Moreover, if we split the trajectories into uj\ = (x t )o<t< s and 0J2 = (xt)s<t<T, for 
any s in the interval [0, T], then the path-probability measure of paths oj\, which 
we just denote by cfP M (wi), is found by integrating over all u>2- 

dV^(m)= I dPtoM (2-4) 

This is an important property. For example, in many computations in this thesis 
we take expectation values of state functions, evaluated at some time s. This gives 

(ffr.)) lla = J dVMf(x s ) = J dV, a (^)f(x s ) 
i.e. we only need to take an average over all paths in the interval [0, s]. 



Particle in a box For Markov chains, a trajectory is determined by the successive 
configurations: uj = (xq, Xi, ■ . ■ , x m ), where m corresponds to the time T. The 
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probability of a trajectory u is just T Xo {u) = n™=o Pn(x n ,x n +i)- If we take for 
example a function f{x) evaluated at time m, then 



E 



n=0 



f(x m ) 



2.3 Probability distribution of states 



The probability of finding the system in a configuration x at time t is denoted 
by Ht(x), the time-evolved distribution (actually in the case of a continuous 
configuration space, fxt(x) is a probability density). It can easily be written in 
terms of expectation values: 



t , t (x) = (S(x t -x)) fio (2.5) 

using 8 Xt ,x instead of 8(xt —x) for discrete state spaces. Equivalcntly, for any state 
function f(x): 

(/(**)>„„ = / dxiM(x)f(x) (2.6) 



The integral becomes a sum for discrete state spaces. In systems with time- 
independent dynamics, there often exists a distribution which does not change 
under the time-evolution. This is called the stationary distribution. If the system 
is in the stationary distribution, it is said to be in the 'steady state'. We always 
denote the stationary distribution with p: 



p(x) = (S(x t - x)) p Vi 

Under certain conditions on the dynamics, it can be proven that such a distribution 
exists, is unique, and that all distributions pt converge to it in the long time limit 
(the system relaxes to the stationary distribution). Unless stated otherwise, we 
always assume this to be true for time-independent dynamics. See |46| for a 
rigorous treatment. 

The existence of such a stationary measure implies crgodicity, meaning that for 
trajectories u> and an arbitrary state function g: 

dtg(xt) -t J dxp(x)g(x) for T ->• oo (2.7) 

almost surely. More precisely: the probability that the system follows a trajectory 
that satisfies (|2.7p is equal to one. 
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Particle in a box For a Markov chain we get: 



Mo(zo) 



L (^m— 1 ) 3?) 



Note that the distribution one time-step later can be written as /j, m +i(x) = 
J2 y Vrn(y)P m (y,x), and thus: 

Hm+i(x) - Mm(z) = ^[^ m (y)P m (y,2:) - (i m (x)P m (x,y)] 

V 

In the case that P n = P, if there is a stationary distribution it has to satisfy 

V 



2.4 The action 



Suppose now that one has two stochastic processes, each having the same 
configuration space but a different dynamics. The action gives a way of switching 
between expectation values computed in those dynamics. This is essential to the 
rest of this text. 



We denote the path-probability measures of the processes by dP and dV* 
respectively. Suppose that dV* is absolutely continuous with respect to dP. This 
means that J M dV*{uS) = for any set of trajectories M for which J M dV(oj) = 0. 
Then one has 

r dV* I dV* \ 

</M); o = J dP^M^M/M - (2.8) 

dV" 

The quantity *° (u>) is called the Radon-Nikodym derivative between the two 
processes [83 . In words: it is the probability of a path in one dynamics divided 
by the probability of the same path in another dynamics. It is common to write 
this in the following way: 



(2.9) 
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where A(lu) is called the 'action.' For Markov processes it is independent of (jlq. 
Normalization of the path-probability measure (|2.3p tells us that 



for any initial distribution ^o- 

Particle in a box For a Markov chain the action is easily computed to be 



2.5 Time-reversal 

To be able to talk about time-reversibility we first define an operator 9 which acts 
as a time-reversal on trajectories. This means that it reverses all trajectories, also 
taking care that velocities change sign under time-reversal: 

w = (x t )o<t<T i-> Ouj = (irx T -t)o<t<T 

where it changes signs of velocities. Moreover, if the dynamics of the process is time- 
dependent, we also reverse this time-dependence. For example if the dynamics is 
governed by a force F t (x), then time-reversal changes this to Fx-t(x). This means 
that the path-probability measure will also change. Therefore, whenever we deal 
with time-dependent dynamics, we denote the new path-probability measure by 
dP R , (the R stands for 'reversed'). These reversals together make the time-reversal 
that intuitively corresponds to playing the movie of a process backwards. 

Let us consider the following measure of irreversibility, defined in terms of a Radon- 
Nikodym derivative: 




(2.10) 




m-1 



5 M (w) = log 




(2.11) 



where fig is an arbitrary probability distribution from which the initial state of 
the system is sampled. For the time-reversed process the initial distribution is 
taken to be fir, which is defined (given /io) through ()2.5[) . This quantity iS^(w) 
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is a measure of irreversibility of a certain trajectory uj. It is the probability of ui 
divided by the probability of its time-reversed twin in the time-reversed dynamics. 
Let us rewrite (|2.11|) as follows: 

S^uj) = log — - — - AT)B 7. M =■ log — -, — r + S[u>) (2.12) 

which defines the quantity S(ui). Note that for Markov processes S(uj) does not 
depend on the initial distribution of the system. 



Particle in a box For Markov chains, time-reversal of the path gives 6u) = 
(x m , . . . , X\, xq), as long as the configurations do not contain velocities, like in 
the example of the particle in a box. The reversal of the dynamics gives transition 
probabilities P n — > P m _„_i. Together they give 



sh = e kg p n ;r Ti (2 - 13) 



2.6 Equilibrium 

The most general way of defining equilibrium is that systems in equilibrium are 
stationary and time-reversible. The first condition means that the probability of 
finding the system in a state x does not depend on time, i.e. it is given by the 
stationary distribution p(x) (the dynamics have to be time-independent of course). 
The second condition means that when we are shown a movie of an equilibrium 
process, we can not tell if the movie is played normally or backwards. More 
mathematically: S p (u>) as defined in (|2.11|) with fj,Q = [At = P is zero for any 
trajectory w. 

^M = log^M=0 (2.14) 

This has as an immediate consequence (see (|2.12p 'l that 

S{lo)= log p{x T )- log p(x ) (2.15) 

We can use our knowledge of equilibrium systems (see Section rO|) to see what this 
means physically. E.g. for a system in contact with a heat bath in equilibrium at 
inverse temperature /3, p is given by the canonical distribution. We get: 



S(w) = -P[U{x T ) ~ U(x )} 



(2.16) 
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with U(x) the energy of the system in configuration x. The change of energy of 
the system is equal to minus the change of energy of the environment. It becomes 
more clear what this quantity S is when we consider a system in contact with a 
particle reservoir at inverse temperature /3 and chemical potential p. For this p is 
the grand canonical distribution, and we get 



This is exactly the change of entropy in the environment, stated in terms of the 
configurations of the system (up to a factor of fee, Boltzmann's constant, which 
we set to one for notational simplicity). This means that S(oj) can be interpreted 
as the entropy flux from the system to its environment during the trajectory uj. In 
the following section we generalize the connection between entropy flux and S(uj) 
to noncquilibrium processes. 

Here we would like to point out that there is a fundamental difference between 
a system in equilibrium and a system with an equilibrium dynamics. As we can 
see from ()2.12p . S(uj) is independent of the initial distribution p , it only depends 
on the dynamics of the process. So if the system has an equilibrium dynamics, 
(I2.15|) holds for any uj, even if the system is not in the stationary (equilibrium) 
distribution. This is also called a detailed balanced dynamics. Only if the system 
is also stationary, then ()2.1ip is zero for any uj and we say that the system is in 
equilibrium. 



Particle in a box For a Markov chain to be in equilibrium, ()2.14p must hold 
for any path. It should therefore hold at least for paths that last only one step: 
uj = (x 0l xi). So a necessary condition for equilibrium is: 



S(uj) 



P[U{x T ) - U(x )} + (3p[N(x T ) - N(x )} 



(2.17) 




for any xq,xi, which is equivalent to 



p(x )P(xo, xi) = p(xi)P(xi, x ) 



This is called the detailed balance condition. One can easily check that it is also 
a sufficient condition for an equilibrium dynamics. 
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2.7 Irreversibility and entropy: local detailed balance 

For a very large class of nonequilibrium systems we will assume that 

S((jj) = the entropy flux from the system into the environment. 

This assumption is called 'local detailed balance,' because it is based on the 
assumption that locally in time and in space the system has a dynamics that 
is detailed balanced. This assumption is restricted to the case that the reservoirs 
only interact with the system, not with each other. Moreover the coupling between 
system and reservoirs should be sufficiently weak and the reservoirs sufficiently 
big, such that the reservoirs stay in equilibrium throughout the process. The 
local detailed balance assumption can be seen as a restriction on the possible 
mathematical models that we use to describe the system. More constructively, it 
gives a partial recipe to write down models that correspond to the physical world. 
Partial, because local detailed balance does not fully specify the dynamics. The 
ideas behind this assumption have been used already for some time, see e.g. |62j . 
but it was first called local detailed balance in [55]. A more rigorous treatment 
of this assumption was then later done in [ftS] [73] [7J] and mostly in [57] . Local 
detailed balance is central to the results discussed in this thesis, and we always 
assume it to be true. 

Let us give a heuristic motivation why such an assumption is reasonable. As said 
in Section 11.31 there are several ways in which a system can be driven away from 
equilibrium. Let us first consider the case of a system in contact with two heat 
baths which are themselves in equilibrium at different temperatures /3i and /3 2 - 
The baths do not interact with each other, only with the system. Microscopically 
the system never interacts with the two baths at the exact same time. Imagine 
therefore that we let the system run for only a very short time-interval [0, T). This 
interval is so short that the system interacts only with heat bath 1 within [0, t], an 
only with heat bath 2 in [t,T]. (A more elaborate discussion of such a process can 
be found in [53J [51] )■ By its mathematical definition ()2.12p , S(u>) is then just the 
sum of contributions from [0,t] and [t, T]. As the dynamics are detailed balanced 
in each time interval separately (there is interaction with only one heat bath at a 
fixed temperature) , we can use the results of the last paragraph to see that 

S(w) = -Pi[U(x t ) - U{x )] - p 2 [U{x T ) - U{x t )) = fcQt + p 2 Q 2 

where Q\ and Q 2 denote the heat fluxes into heat bath 1 and 2 respectively. 
Therefore, S(uj) is the change in entropy of the environment due to the process of 
the system, i.e. the entropy flux from the system into the environment. Given an 
arbitrary time-interval, it is therefore reasonable to assume that we can split this 
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interval into many small intervals which are short enough so that we can assume the 
dynamics to be detailed balanced in each interval separately: we assume detailed 
balance 'locally in time'. Adding all the contributions to S(ui) from these small 
intervals gives us that 

S(w) = jSiQi + ftQa 

Using the same reasoning we can expand this formula to the case in which the 
system interacts with several heat baths and particle reservoirs. The conclusion 
stays the same: the quantity S(co) is the entropy flux from the system into the 
environment. 

Finally, a system can be driven from equilibrium by some nonconservative forcing. 
Let us illustrate this last class of systems with an example: 

Particle in a box Consider the following example: the box in which the charged 
particle resides is surrounded by an environment at equilibrium at a single 
temperature (air or water). The air inside the box is also at equilibrium at that 
temperature. Suppose the box has the shape of a thin torus, with a circumference 
L, which we divide into TV segments of length d (d would be the error of our 
measurement). The configuration space is thus a ring with N sites, labelled 
x = 1,2, . . . , N, N + 1 = 1. We assume that the particle can maximally move one 
site to the left or right during one time step. An electric field E (constant in time) 
is applied on the box. As a consequence, the particle gains an energy E(x, y) ■ d 
when going to the left or right (y = x ± 1). Note that E(x, y) = —E[y, x). 

First of all, suppose that the electric field is of the form E(x, y) ■ d = U(y) — U(x), 
meaning that U(x) is the energy of the particle at site x. In this case the forcing is 
conservative, and the dynamics of the system should satisfy the detailed balance 
condition: 

P ( X >y) _ p -f}[U{y)-U(x)\ _ J3E(x,y)-d 

P(y,x) 

If we restrict our attention to only two sites x, y, it is impossible so say if the electric 
field is conservative or not. For a nonconservative force we therefore assume that 
detailed balance holds locally (i.e. between each pair of states separately): 



P( x i V) _ j3E(x,y)-d 

P(y,x) 
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The quantity S(uj) is then given by (|2.13D : 



m— 1 



S(u) = y, log 



n=0 



which is the entropy production in the environment (the entropy flux into the 
environment) . 

In general, note that the word 'conservative' is a global property. If one considers 
a small enough region of the configuration space, one can always define a 
local potential from which the force is derived. So restricting our observation 
momentarily to this small region, we assume that the dynamics of the system is 
detailed balanced (detailed balance locally in space), and still conclude that S{uS) 
is the entropy flux into the environment. Returning to the global picture, the sum 
of all these small regions gives then the total entropy flux into the environment. 
The only difference with a real detailed balanced dynamics is that this entropy 
flux is no longer the difference of a state function, but depends on the whole path 

UJ. 



It turns out that we can also attach a physical meaning to S^{u), as defined in 
(|2.1ip . when the average is taken. This means again an average over all possible 
trajectories ui = (xt)o<t<T- 



The second term on the right-hand side is the average entropy flux into the 
environment. The first term can be rewritten, using the fact that for any state 
function /, we have that (/(£t)) W) = / dx^ t (x)f(x) (where the integral becomes 
a sum for Markov jump processes) . We see that the average of becomes 



2.8 Average entropy 




s(mt) - s(/io) + (S(oj)} 



(2.18) 



with s(^) the Shannon entropy of the distribution (i 




(again up to a factor of ks)- When /i is an equilibrium distribution, the Shannon 
entropy is equal to the Gibbs entropy, which is in equilibrium statistical mechanics 
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the (physical) entropy of the system. Therefore we still call it here the entropy 
of the system, making the average of equal to the average of the total entropy 
change in the world due to the process of the system. 



The average entropy change is positive. This is a direct consequence of its 
definition, 



H = dp„»_£^( w ) = l 



dV / dV 

u ' Mo / J Mo 



Mo 

and Jensen's inequality: 



1 = (e~ s ") > 

\ / tin — 



Mo 



2.9 Entropy and traffic 



In (|2.9p we wrote the Radon-Nikodym derivative, which compares the path- 
probability measures of two different dynamics. This defined an action A(lo). 
Similarly, we can define an action for the time-reversed process: 

rW*-R 
Mo 

As an immediate consequence of the local detailed balance assumption, we see that 
A r (0oj) - A{lj) = S*(lj) - S(w) = S ex (oj) (2.19) 



In words: the time-antisymmetric part of the action A is equal to the difference in 
entropy fluxes in the two dynamics. We call it an excess entropy flux. Here 
we clearly sec the limits of local detailed balance: it only specifies the time- 
antisymmetric part of the action. And, in contrast to equilibrium dynamics, 
the time-symmetric part does play an important role in statistical physics out of 
equilibrium. Therefore it has been proposed [11, 22 , 68 to examine the following 
quantity more closely: 

T ex (u) - T(w) - T(w) = A R (6ui) + A{u) 

The quantity T(w) is called traffic [69j ITU |70] , and the time-symmetric part of the 
action is therefore an excess traffic. The action A can thus be written as 
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and the Radon-Nikodym derivative: 



The physical and operational meaning of traffic is up to now not completely clear. 
From the research discussed in this thesis, it seems to be of great importance in 
results of nonequilibrium statistical physics. 



Using as an assumption only local detailed balance, a number of well-known results 
can be very generally derived. We conclude this chapter with these derivations. 

2.10.1 Fluctuation theorem 

A recent and celebrated result of out-of-cquilibrium statistical physics is the 
fluctuation theorem for entropy fluxes [SB] [321 IH] , valid for systems with a time- 
independent dynamics: 



where P (S(co) = To) is the probability density that the entropy flux is equal to 
a value To. We use local detailed balance to relate entropy fluxes to the path- 
probability measure. With this the derivation is not difficult. We first consider 
the probability density for S p (ui) (started from the stationary distribution p), and 
we let the process run for a finite time T. 



where, as before, the integral is taken over all possible paths uj = (xt)o<t<T- Of 
course, computing this distribution is not possible in general, but we can rewrite 
it: 



2.10 General physical results 




(2.21) 
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Where in the last line, we used the definition of S p (oj) and the fact that the delta 
function makes sure that S p (uj) = To. We now make a change of variables uj — > Qui, 
which is an involution. Note that this changes S p to — S p : 

P{S p {ta)=To) = e Ta J dV p {uj)5{S p {Lo)+Tn) 

= e Ta P (S p (us) = —To) 

This is what is called a fluctuation theorem for finite times for the quantity S p . 
However, the local detailed balance assumption only says that S(oS) is an entropy 
flux. Without going into mathematical rigour, we do expect that, typically: 

T^oo T T^oo T 

because the difference between S and S p is only in temporal boundary terms. 
With this we get the fluctuation theorem (|2.21[) . Note that it is possible that the 
boundary terms stay important even in the long time limit. In such cases the 
fluctuation theorem is not valid, see [90] . 



2.10.2 Work relations 

Noncquilibrium work relations [TU HOI El] can be derived for a system connected to 
a single heat bath, with a time-dependent dynamics parametrized by a parameter 
A*. This parameter is called the protocol. Of course, because At changes in time, 
the system is pulled out of equilibrium. Usually it is assumed that for each fixed 
value A the dynamics is detailed balanced. Actually, to have work relations, the 
dynamics only needs to be detailed balanced at the beginning and the end of the 
process. 



We start at time zero with the system prepared in the equilibrium distribution 
corresponding to the value Ao = A. We then let the process run in the time- 
dependent dynamics until time T. During the process there is local detailed 
balance, but at time T there is again a detailed balanced dynamics with At = B. 
Consider the following quantity: 



*P*6 K >~ P b(xt) 



where S(u>) is by the local detailed balance assumption equal to /3 times the 
heat flux Q(ui) into the environment . Remember that with the time-reversal, 
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also the protocol A t changes to At-* (see Section 12. 5p , hence the superscript 
R. Substituting the explicit expression of the equilibrium distributions (pa(x) = 
_^_ e -PU(A,x) an j s i m ii ar iy f or \ = B), we get 

d^PA = e P[FA-F B -U{A,x )+U(B,x T )+Q(u>)\ _ e 0[-AF+W(w)] 



dV R 



where AF — Fb — Fa = ^-[logZ^ — log Zb] is the change of free energy between 
the equilibrium states corresponding to the values A and B of the parameter A, 
and W is the work done on the system. 



From this basic result a work relation can be derived, which was done for the first 
time by Crooks [THJUO]: ^ P{W = w) be the probability density that the work 
during the process is equal to w. Then 



P(W = w) = I dV PA (w)J(W(w) - w) 

dV^ B (6uj)e^- AF + w( ^S(W(uj) - w) 

e f3[-AF+ W ] f dv R ( U )S(W(W)+W) 



= e^- AF +^P R (W = -w) 

where we defined P R (W = —w) as the probability density that the work during 
the reversed process is equal to — w, and we used that W(9uj) = —W(ui). From 
this relation the following equality can easily be derived (obtained by Jarzynski in 

my- 

{e- pw ) pA = I ' dwP(W = w)e- pw 



J dwer fiAF P R {W = -w) 



e -,3AF 



which relates noncquilibrium work- values to equilibrium free energies. Again we 
see that these results are an immediate consequence of the local detailed balance 
assumption. 
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2.10.3 McLennan formula 

The McLennan formula [7HJ HO] gives an approximation of the stationary measure 
for a dynamics that is close to detailed balance. For a system in contact with two 
heat baths (particle reservoirs) close to detailed balance means that the difference 
in the temperatures (chemical potentials) is small, while for a system under the 
influence of a nonconscrvative force this means that the force is small. In general, 
we can parametrize the 'distance' from detailed balance by a parameter e, (e.g. 
e = T2 — T\ or the nonconservative force F = ef). In the case that e is small, one 
can expand the stationary distribution around e = 0. Up to first order in e this 
gives the McLennan formula. 



We derive the McLennan formula in the following way: suppose that until time 
t = the system is detailed balanced (e = 0), and at time t = the system has the 
corresponding equilibrium distribution fig — p a . At time zero, we drive the system 
from equilibrium, parametrized by e. After a time t the probability distribution 
of being in a state x is then given by 

Ht{x) = (S(x t - x)) c p0 

where the average is an average over all possible paths in the interval [0,t], 
computed in the dynamics with e (hence the superscript). To rewrite expectation 
values in the noncquilibrium system into expectation values in the equilibrium 
system, let us write down the Radon-Nikodym derivative (|2.9p : 

dry 10 ' 

so that the probability distribution becomes: 

with the superscript denoting that the average is taken in the equilibrium process. 
The expectation value of an observable Q(to) in an equilibrium process, is the same 
as that of the time-reversed observable Q(9uj). This is a simple consequence of 
(I2.14[) . This means that we have 



IH{x) = (S(x - x)e- A ^) C [ 

= (s(x ~x)e- s ^e- A ^y o 
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= (S{x - x)e- s "My Q 

where the excess entropy flux S ex — S c — So is the difference in entropy fluxes 
in the nonequilibrium and the equilibrium system (see ()2. 19|) ) . One can see from 
the definition of S ex that it is zero when e is zero. Moreover, in physical systems 
(and certainly the ones we discuss in this thesis) , S ex is at least of first order in 
c. lim £ _yo S ex j ' e < oo. An expansion in e gives 

Ht(x) = p°(x) - (S(x - x)S ex (u>))° p0 + o(e) = p°(x)[l - (S e x(u)) x ] + o(e) 

where the last expectation is an average over all trajectories starting from the state 
x. Up to first order in e this formula gives the probability distribution of finding 
the system in state x at an arbitrary time. Letting the time go to infinity, this 
will converge to the stationary state p e (x) corresponding to the nonequilibrium 
dynamics. 

2.10.4 Fluctuation-dissipation theorem in equilibrium 

The equilibrium fluctuation-dissipation theorem [131 159] tells us how systems 
respond to a small change in their Hamiltonian (energy), at least in the linear 
regime. The framework in which we work here is almost the same as for the 
McLennan formula: up to time zero the system enjoys an equilibrium dynamics 
with a Hamiltonian Hq, and at time zero it has relaxed to the corresponding 
equilibrium distribution p cx exp(—/3Ho). At time zero an extra potential is added 
to the dynamics: H = Hq — JiV, where h is a small parameter. For finite times after 
this perturbation has been made, the system is not yet in equilibrium. We want to 
compute the expectation values of observables in this perturbed (nonequilibrium) 
system. We consider only observables that are state functions, denoting them by 
Q(x). With a reasoning completely analogous to the one made for the McLennan 
formula (the only difference is that the observable S(xt — x) is replaced by Q(xt)) 
one then finds that: 

{Q(x t )) h p0 = (Q(x )e- s °*M) H p0 

where the superscript h denotes that the averages are taken in the perturbed 
system, and S ex is the excess entropy flux, excess of the perturbed system versus 
the unperturbed system. As the only difference between the two dynamics is the 
addition of the potential hV, this excess must be equal to (3h[V(xt) — V(a;o)]. 

In linear perturbation theory one is restricted to systems for which such a small 
perturbation has only a small influence. One can therefore work up to linear order 
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in the small parameter h: 

(Q(Xt)) p o = (Q(x ))° p a - (Q(xo)Sex(u))° p0 + o(h) 

= (Q(*o)>°c + (Q(x t )S ex (uj)) p0 + o(h) 

In the last step we have again used that in an equilibrium dynamics the 
expectations of observables and their time-reversals are the same. Using the 
explicit expression for the excess entropy flux, we get 

= 0(Q(xt)\y(xt)-V{x o )]) O f p (2.22) 

h=0 

which is the fluctuation-dissipation theorem in equilibrium, it is valid for any 
time t. The left hand side of this equation is called a response function. It is the 
response of the expectation value of an observable to an added potential and is 
thus equal to the correlation of the observable with the extra entropy flux created 
by the potential. 

The fluctuation-dissipation theorem is useful because it gives a relation between 
two quantities in essentially different processes. One can e.g. determine the 
response of a system without actually perturbing it. Part of the results in this 
thesis are about the generalization of this relation to noncquilibrium systems. 



Chapter 3 

Markov jump processes 



Markov jump processes are Markov processes where the configuration changes 
discretely. This means that in a finite time-interval, the configuration changes 
a finite number of times. In this thesis we only work with discrete configuration 
spaces where Markov jump processes are concerned. Such processes are widely 
used in physical modelling because they are relatively simple, but approximate real 
physical systems guite accurately. The discrete configuration space can arise as a 
result of a discrete approximation of a continuous space, or because of quantum 
mechanical principles (e.g. spins in a magnetic field). We give a brief introduction 
to Markov jump processes here, focusing on the properties relevant for the rest of 
the thesis. For a thorough introduction, we refer to \46]! - As a motivation, we start 
with a very physical example. 



3.1 An introductory example 

Imagine a chemical reaction where two reactants A and B react to form a product 
C. The reaction is facilitated by the use of a catalyst X as follows: first the 
molecule A binds to the catalyst, forming a new molecule Xa- Then B binds, 
forming Xab- When bound to X, A and B react and form Xq- Finally C 
decouples from the catalyst, leaving the catalyst in its original state X, see Figure 

ixn 

The details of the reaction A + B — !> C thus depend strongly on the reaction cycle 
of the catalyst X. Apart from measuring the average speed or rate of such a cycle, 
we are also interested in more detailed information. What are the fluctuations from 
this average speed? Does the inverse cycle also occur, and with what probability? 
For such questions a more detailed model of the reaction is required. As it is 
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reactant A 



product C 




"AB 



reactant B 



Figure 3.1: A catalytic reaction cycle. 



impossible to predict all the precise positions and velocities of all the molecules, 
a Hamiltonian description is not wanted. Therefore we use a stochastic model. 
One can imagine all the molecules involved in the reactions are 'wandering' in 
some solution. A reaction can occur when two molecules meet each other. The 
speed or rate of each step in the reaction cycle then depends on parameters as 
concentrations of the molecules in the solvent, temperature, and of course also on 
the probability that, given that two molecules meet, they actually react. If the 
concentrations of the molecules in the solvent are not too big, it is a rather good 
assumption to think of this process as being Markovian. Indeed, we assume that a 
reactant molecule makes many collisions with the solvent molecules before meeting 
with another reactant, thus effectively losing its 'memory' on a timescale that is 
much smaller than we are interested in: the timescale of the successive reactions. 
Also, this allows us to treat the molecules of the catalyst as independent particles. 

To make a model of the reaction cycle, we therefore take one catalyst molecule 
as the system of interest. This molecule can be in four different chemical states: 
X, Xa, Xab and Xq- At any time, regardless of its history, the molecule can 
change its state as a consequence of a reaction. We thus arrive at a Markov 
process on a finite configuration space, but in continuous time. This fits exactly 
in the framework of Markov jump processes. 

This is only one example where Markov jump processes provide a good model. 
Other examples are traffic jams, transport of ions through nanotubes, the Ising 
model and other models of spins in a magnetic field, population dynamics in 
ecological systems, etc. Providing important and relevant models in physics, 
Markov jump processes are used throughout this thesis. In this chapter we 
therefore introduce the aspects relevant for the rest of this thesis. For a thorough 
introduction, we refer to |46j . At the end of this chapter we return to this model 
and see how to describe it as a Markov jump process. 
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3.2 Definition 

We work here with a discrete, even finite, set f2 of configurations x, y, . . .. The 
dynamics of a Markov jump process is defined as follows: let P At (y\x,t) denote 
the probability that the system changes (jumps) from configuration x to y =/= x 
within [t, t + At], given that the system was in configuration x at time t. Then 

P At (y\x, t) = k t (x, y)At + o(At) (3.1) 

The k t (x,y) arc called transition rates. Obviously k t (x,y) > 0. Also, as a 
convention, we take k t (x,x) = 0. From (|3.ip one also sees that the probability to 
jump two times within a time span At is of order o(At). Logically, the probability 
P At (x\x, t) that the system does not jump within [t, t + At] is then 

P At (x\x, t) = 1 - \ t (x)At + o(At) with X t (x) = h{x, y) (3.2) 

This Xt{x) is called the escape rate, as it quantifies the probability that the 
system 'escapes' from x. We assume that e < X t (x) < X for all t,x and for some 
e > 0, A < oo, to make sure that the expected time for the system to jump is finite. 

In Figure 13.21 an example of a typical realization of a Markov jump process is 
shown: the system stays in a certain configuration for an exponentially distributed 
time (we prove this later on in this chapter), and then jumps to the next state. 
We adopt the convention that at the jump time, the system is in the state after 



1 1 

■f • o 

•- 



t 



Figure 3.2: A realization of a Markov jump process. 



the jump, i.e. in Figure [5721 the graph is right-continuous. 
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3.3 Probability distributions and the Master equation 

Suppose that we know the probability that the system is in state x at time t: 
fj.t(x), for all x. From this we can compute the probability that the system is in 
any state x at time t + At. It is equal to the probability that the system was in 
state x at time t and stayed there, plus the probability that it was in any state y 
and jumped to x. With (|3.ip and (|3.2p it is clear that 

IH+m(x) = IH{x)[l - X t (x)At + o(At)] + J2 Mv)Mv, x)At + o(At)] 

This gives us a differential equation for [i t '- 

= Hm^M^M = J2MyMy,x)-^(x)k t (x,y)] (3.3) 

This equation is called the Master equation. Notice that it is a deterministic 
equation: all uncertainty is in the probability distribution fi t , while the evolution 
of [it through time is deterministic. 



The quantity jp, t (x,y) = p,t{x)k t {x , y) — IM{y)kt{y , x) can be seen as the current 
between x and y, or better even: a probability current. Indeed, the first term is 
the probability per unit of time that the system makes a jump from x to y, while 
the second term is the probability per unit of time that the system makes a jump 
from y to x. With this definition of current we can rephrase the Master equation 
as a continuity equation: 



dnt(x) 



dt 



Y,M*,y) = o (3.4) 



3.4 Stationarity and detailed balance 

In the special case that the transition rates of the Markov process are independent 
of time (k t (x,y) = k(x,y)), we call the process time-homogeneous. As said 
before, for time-independent dynamics we assume that there exists a stationary 
distribution p, which is unique, and that all distributions fi t converge to it in 
the long time limit. This p solves the Master Equation (|3.3p with the left-hand 
side zero. The conditions under which this assumption is valid are conditions of 
irreducibility: a jump process is called irreducible if the probability of the system 
to go from any state x to any other state y in a finite time is non-zero. This 
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is equivalent to saying there is a chain of states x = x\, X2, ■ ■ ■ , x n = y such that 
k(xi,x i+ i) > for alii = 1 to n. The proof of this is beyond the scope of this text. 

Finding the stationary distribution is difficult in general. There is however a 
special case in which computations simplify: if there exists a function f(x), (/ 7^ 
and f(x) > 0), such that 



then p(x) tx f(x) solves the stationary Master Equation. If the transition rates 
satisfy this property, then the process is said to satisfy detailed balance, and 
p is called the equilibrium distribution. Once the system has relaxed to this 
distribution, we then also have that j p (x, y) — 0, so that in equilibrium all currents 
between different states are zero. This definition of equilibrium coincides with the 
one given in the previous chapter. 



To have a better idea of how the system evolves from configuration to configuration, 
we compute the following probability: for any t > s, let P(y, t\x, s)dt be the 
probability that the system remains in configuration x until it jumps within [i, t + 
dt] to state y, given that the system was in configuration x at time s. Then 



We prove this as follows: divide the interval [s, t) into n pieces of length At = — -. 
Then from (|3.1[) and (|3.2p we see that 



f(x)k(x,y) = f(y)k(y,x) 



(3.5) 



3.5 Finite time probabilities 



P(y,t\x,s)dt = k t (x,y)e J' 



dt 



(3.6) 



n-l 



P{y, t\x, s)dt 



k t (x, y)dt J[ [1 X s+kAt (x)At + o{At)} 





If we now let n — >• 00 , we get exactly (|3.6[> . 
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Let us examine this probability: the probability to jump to anywhere within 
[t, t + dt], given that the system was in state x at time s is 

P{t\x, s)dt = P(V, t\x, s)dt = X t (x)e~ SI x ^ du dt 
y 

On the other hand, given that the system jumps at time t, the probability that it 
jumps to state y is then 

P(y,t\x,s)dt _ k t (x,y) 
Pt[X ' y> P(t\x,a)dt ~ X t (x) 

The Pt{x, y) are called the transition probabilities, and we see that ^2 y ^ x Pt(%, y) = 
1. In conclusion, when at time s the system is in a state x, it stays there for a 
time t — s that is exponentially distributed, determined by the escape rate At (a;). 
When it jumps, the probability that it jumps to a state y is pt(x, y). 



3.6 Trajectories 

A trajectory to = (xt)o<t<T is completely described by giving the consecutive 
states Xi (with i = 0, . . . n) the system visits plus the times ti at which it jumped: 

x t = Xi for ti <t < U+i 

where to = and t n is the last jump time before time T. Using (|3.6[) . we can 
compute the probability measure of such a path: 

n-l 
i=0 

The last factor on the right-hand side is the probability that the system does not 
jump in the interval [t n , T) . We rewrite this path-probability measure in a slightly 
more elegant form: 



I \ - f t ' + 1 ^(x,)du -J X u (x n )du 

^{Xi^i+ije Jt > dt l+ i e J '» 
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dV^ (cj) = fj,o(x ) exp \ log kt i+1 {xj, x i+ i) - / \ t {x t )dt > dh . . . dt n 
{ *=o Jo J 

(3.7) 

With this path-probability one can then compute expectation values, as defined in 
(|2.2p , and the time-evolved probability distribution of configurations [i t , as defined 
in (|2.5|) . This definition of fit is completely consistent with the Master equation, 
of course. 



3.7 The Girsanov formula 

Suppose now that one has a different Markov jump process with rates k*(x,y) 
and one wants to calculate the expectation values of observables in this process. 
Then one can relate this to the expectation values in the original process by using 
the Radon-Nikodym derivative (see (|2.8|) and (|2.9|) V The demand of absolute 
continuity boils down to the demand that k*(x, y) ^ for any pair x, y for which 
k(x,y) ^ 0. It follows immediately from (|3.7p that 

e-*"> = exp(|> ( f^ X ^\ ) + rdtlX^-A;^)}) (3.8) 

[ i=0 \ K ti+i \ x h x i+l )/ Jo J 

with the action defined as in (|2.9|) This is often schematically written as 

A(u) = dt[X* t ( Xt ) - X t (x t )] gin (3-9) 

where the sum is over jump times, and t~ is the time just before the jump. This 
formula (|3.9p . the Radon-Nikodym derivative for Markov jump processes, is called 
the Girsanov formula, and is of much use throughout this text. More details can 
be found in Appendix 2 of [57]. 



3.8 Local detailed balance 



The Markov jump processes as described up to now are little more than 
mathematical models. To make a connection with physics we make the local 
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detailed balance assumption: the quantity S(ui) that was defined in (|2.12p , is equal 
to the entropy flux into the environment during the trajectory oj. For Markov jump 
processes this becomes (see (|3.7|) ) : 

Qf A 1 ( dVx o I \\ V^l k t (x t -,X t ) 

«= lo H^ H J = g los M^) 

We see that there is only an entropy flux when there is a jump, and the total 
entropy flux is just the sum of entropy fluxes for each jump: 

where <Jt{x, y) is the entropy flux associated to one jump from a state a; to a state y. 



Example 1: As a first example we imagine a system with time-homogeneous 
dynamics, immersed in a single heat bath at inverse temperature f3. We denote 
the energy of the system, (which depends on the configuration x) as U(x). On 
top of that there is a forcing, meaning that for every jump from a state x to 
a state y, there is an amount of work W(x, y) needed. Note that by definition 
W(x, y) = — W(y, x). Using the first law of thermodynamics, we see that 

k ( x >V) = e -P[U{y)-U(x)-W{x,y)} /g lQ \ 

k(y,x) 

Indeed, local detailed balance allows us to express a part of the transition rates in 
terms of physical, measurable quantities. 

If there exists a state function V such that for all x, y we have W(x, y) = V(y) — 
V(x), then the forcing is conservative, and by (|3.10p the rates are detailed balanced 
with the corresponding equilibrium distribution p(x) = cxp{— (j\U{x) — V(x)]}. 



Example 2 : As a second example think of two big particle reservoirs with different 
particle densities. Between these reservoirs there is a wall, with a narrow channel 
going through it from one side to the other. The channel is so narrow that particles 
are effectively restricted to one dimension. This is what happens for example in 
ionic transport through cell membranes. Because of the difference in particle 
densities a current arises towards the side with the lower density. 

We model the channel as a set of sites labelled by % = 1, . . . , N. On each site 
there can either be one or no particle. A configuration of the system is thus an 
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array x, with empty sites xi = and particles x% = 1. Each particle can hop to the 
neighbouring site left or right if that site is unoccupied. Such a process is called an 
exclusion process. Hence the only transitions allowed are x —> x^' where j = i± 1 
and 



£fc if i^k^ j 

Xj if ft = i (3.11) 
Xi if fc = j 



An exception occurs at the boundaries of the system i = 1,N, where particles 
can enter and leave the system. Hence transitions x — > x^ are also allowed, with 
i = 1 , N and 



■r i 



(i) _ J a^fc if fc 7^ * 



fe I 1 — Xk if k = 



(3.12) 



This means that at the left and right boundary the system is in contact with 
particle reservoirs, which are characterized by a temperature and a chemical 
potential. In Figure 13.31 an example of such a process is visualized. This is an 
example of Kawasaki dynamics. 



Physically, we assume here that there is no interaction between the particles, 
except for the exclusion. We take therefore the energy of the system zero. The 
particle number of this system is not conserved however. If both reservoirs have 
the same inverse temperature (3 and chemical potential fi, the system is detailed 
balanced, and for any two states x, y: 



k(x,y) 
k(y,x) 



e Pp,[n(y)-n(x)] 



with n(x) the number of particles in configuration x. We see from this that when 
a particle hops within the system, the transition rate is symmetric: k{x,x^') — 
k{x^\x). Detailed balance gives us as equilibrium distribution 



p(x) 



and the average particle density d 



n(x) _ 

~W ~ 1 + e^ 
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This d is the particle density of the particle reservoir, as it is equal to the density 
of the system when in equilibrium. 

If the chemical potentials of the two reservoirs are not the same, we use local 
detailed balance. We get our inspiration from the detailed balance case to write 

for events where a particle enters or leaves the system (i = 1, N), and k(x, x^) = 
k{x^\x) for events where a particle hops within the system. Again inspired by 
the detailed balance case we define the particle densities of the reservoirs as 



di = 



eft* 



d l l + l I l + I I I • d n 

1 2 n 



Figure 3.3: A visualization of an exclusion process. 

Finding the stationary distribution in this case is possible, but certainly not trivial. 
We refer to |5S] for more details. We can say something about the current though. 
For this we use the fluctuation theorem discussed in Section B.10.11 For any specific 
u> we can write, using (|3.13p 

S(uj) = I3T^iJi(uj) + /3Tfi N J N (uj) 

where the current Jiiuf) is the net number of particles per unit of time that has 
gone into the system through site i . Note that the sum of these two currents gives 
the change of particle number in the system per unit of time: J\{lo) + Jn(^>) — 
[ti{xt) — n(xo)]/T. Therefore: 

lim [Ji(u) + Jjv(w)] =0 

T->oo 

In the limit of large T we therefore just use J = J\ = —Jn- The fluctuation 
theorem ()2.21[) can then be reformulated as 

T limIlog(^^)=^ 1 -M 
T^ooT \P{J{w) = -3)J 
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The local detailed balance assumption on which the fluctuation theorem is based 
thus predicts what is the most probable direction of the particle current. Not 
surprisingly the most probable direction of the current is towards the lower 
chemical potential. The fluctuation theorem asserts that the reverse current is 
exponentially less probable. 

Example 3: Let us return to the introductory example from the beginning of this 
chapter. More precisely: £1 consists of 4 chemical states , labelled by x = 1, ... ,4 
and lying on a ring, i.e. 5 = 1. These states correspond to: 1 = X, 2 = Xa, 
3 = Xab and 4 = Xc- We can see that k(x, y) only differs from zero for \y — x\ = 1. 
Let us denote by U(x) the energy of the molecule when it is in chemical state x, 
and [ii and iVj are the chemical potentials and particle numbers of the reactant 
particles in the solvent, where i = A, B,C. These reactant particles arc seen 
as the reservoirs of our catalyst molecule. As we imagine the reservoirs to be 
in equilibrium, their chemical potentials are fixed. The entropy change in the 
environment for a transition x — > y can thus be written as 

a(x, y) = -j3[U(y) - U(x)} - /3 ]T ^AN Z 

i 

where ATV^ is the change in particles of species i. For example, if the catalyst 
molecule makes a transition X -> Xa, we get as entropy flux to the environment: 

log = 2) = -mm - u(i)} + p»A 

It is the chemical potentials of the 'reservoirs' that drive the system from 
equilibrium. To show this, we take as trajectory one reaction cycle X — > Xa — > 
Xab — > Xc — > X. We can compute the entropy flux into the environment during 
this cycle: 

4 

^ 0-(i, i + 1) = P[ha + [i-B - fJ-c] 

i=l 

For an equilibrium dynamics this entropy flux should be equal to zero (see (|2.15|) ) . 
Whenever [i c ^ [i A + Ms> the probability that the catalyst covers the reaction 
cycle in one direction is not equal to the probability for the other direction. In 
fact, when fic < + Mb the reaction cycle will on average go in the direction as 
in Fig. ED 



Chapter 4 

Diffusions 



Langevin dynamics (diffusions) are well-known and widely used as a combination 
of Hamiltonian dynamics and stochastic noise. Mathematically they are more 
difficult to define than Markov jump processes, but their physical meaning is much 
more clear, as the dynamics are expressed in terms of measurable quantities as 
forces and positions, or electric fields and currents. One distinguishes in diffusions 
between the underdamped (inertial) regime and the overdamped regime. In the 
latter accelerations are ignored due to high friction. In this chapter we briefly 
introduce diffusions, treating both the underdamped and the overdamped case. For 
the interested reader, we suggest \18l \58l I7ffl \81f . 



4.1 The Langevin equation 

For didactic purposes we start with the simplest case of one test-particle in one 
dimension. The state space is here the phase space, meaning the space of the 
possible positions and velocities of the particle. Pure Hamiltonian dynamics would 
give two equations of motion: 

dxt = vtdt 

mdv t = F t {x t )dt (4.1) 

which correspond to a particle that experiences a deterministic force F t . Note that 
m is the mass of the particle, Xt its position at time t and Vt its velocity. Now we 
put the particle in a heat bath (imagine a glass of water) at inverse temperature /3. 
The particles of the heat bath will start colliding with our test-particle. Because 
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we do not know all the exact movements of these particles we model the effect 
of these collisions as a random force on the test-particle. The modified equations 
then look like this: 

dx t = v t dt 



and are called the Langevin equations. The term —mjvt is the frictional force that 
the test-particle feels, with 7 the friction coefficient. Indeed, when the particle has 
a velocity, then it has more head-on collisions than from behind, with a net effect 
equal to —mrfOt- This friction force is an average effect of the heat bath. That is 
why still another term is added, namely \ / '2T>dB t , with T> the diffusion coefficient. 
This should model the random kicks the particle receives. B t is what is called a 
Wiener process. It is a random variable with the property Bt — B s ~ 7V(0, t — s), 
meaning that it has independent increments with a Gaussian distribution with 
mean zero and variance t — s. This is the reason that dBt is usually called a 
standard Gaussian white noise. The random variable £t = is often used here, 
but mathematically ill-defined. It also has mean zero, but has the property that 
(£s£t) = S(t — s). This means that we have approximated the physical process of 
collisions with particles of the bath, as a succession of infinitely many, infinitely 
big kicks. As a consequence, dvt is not really well-defined. Still the equations (|4.2[) 
are mathematically well-defined if one integrates them with respect to time. This 
is not done here because the Langevin equations do have a clear physical meaning 
in this form ()4.2|) . 

An important remark for things to follow is that we interpret dvt in the Langevin 
equation as Vt+dt — Vt, and dxt analogously. This is called the Ito-interpretation. 
It is also true that the inverse temperature /3 of the heat bath has not yet entered 
the description. We come back to this at the end of this chapter, when we discuss 
the local detailed balance assumption. 



4.2 The Fokker-Planck equation and stationarity 



We sample the initial positions and velocities (at time zero) from an initial 
probability density fio(x,v). One can then derive (see Appendix |B|) from the 
Langevin equation an evolution equation for the densities at later times, which is 
called the Fokker-Planck equation: 




(4.2) 




v d 2 vt 



(4.3) 



m 2 dv 2 
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Just as in the case of Markov jump processes, this evolution equation is 
deterministic, and we can rewrite it in the form of a continuity equation: 

^ + V-^=0 (4.4) 
with a probability current defined by 

3nt = v^t, ( jv )n t - — (4.5) 

\ m m z ov J 

When the force is time-independent, we assume as always that there is a stationary 
distribution p, (which solves the Fokkcr-Planck equation with left-hand side zero). 
Unfortunately, it is often impossible to just solve the Fokkcr-Planck equation. 

In the special case however that we have F(x) — — 9 u ^ , then the solution is 
immediately found to be 

Zj 

where /3 = Indeed, in this special case the particle is in equilibrium with 

the surrounding fluid at inverse temperature (3 and has the well-known Gibbs 
distribution, giving the probability of a state (x, v) as the exponential of minus (3 
times the energy of the state. There is one difference with the Markov jump case, 
namely that the probability current in equilibrium is not zero here: 

/ 1 dU 

On the other hand, this is equal to the current one would have in a purely 
Hamiltonian (deterministic) dynamics, so one could say that in equilibrium, the 
diffusive part of the current is zero. 



4.3 Trajectories 

The position and velocity of the particle evolve according to the Langcvin equation, 
and thus follow a trajectory w through phase-space: u = (x t ,v t )o<t<T- One can 
compute the probability of such a path. We give a heuristic derivation of this, for 
a mathematically rigorous treatment one should consult |65j . 
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Given the initial position and velocity of the particle, the trajectory uj is completely 
determined by the increments of the Wiener process. Let us consider the case 
that xt and Vt are given, and compute the probability of having (xt+dt,vt+dt) an 
infinitesimal time later. We know from the Langevin equation (|4.2|) that 

dB t = \mdvt — FAxAdt + rwyv+dt] 

And the fact that dBt = Bt+dt — Pt ~ A/"(0, dt) means that we have a probability 
density P given by: 

dP{v t+ dt,Xt+dt\xt,v t ) = P(v t+ dt,xt+dt\xt,vt)dx t dvt 

= jj5(x t+dt -x t - v t dt)e- L ^' dv ^dx t dv t (4.6) 

where L is given by 

L(xt,v t ,dv t ) = * [mdvt - F t (x t )dt + rwyvtdt] 2 
= ^i m ^jj- ~ F t(xt) + mjvt} 2 



Note that N = ^ 4 ^ dt is a normalization constant. Furthermore we know that 
xt+dt — Xt + Vtdt, which is why a delta function is inserted. From now on, we 
restrict the space of all trajectories uj to those trajectories which satisfy this 
constraint. This means that the delta function is no longer necessary. Let us 
now divide the interval [0,T] into n intervals of length At (so nAt = T). We 
should take At small enough, so that (|4.6p can be used. (We take the limit of 
At — > 0). Define i; = iAt for i = 0, . . . , n, then the probability density of uj is just 
the product of the probability densities of all the steps Uj i — > Vt i+1 , because the 
increments of a Wiener process arc mutually independent: 



Ho (x ,vq) j 1 \i v ti+1 - Vtj . > 
V M = ^ cx P)-^l^ At i m At Ftixt^ + mjvt 



which we denote in the limit of At — >• by 



li (x ,v ) J 1 



^mo( w ) = jf CX P S I dt[mv t - F t (x t ) + rwyv t 



T 



12 



TRAJECTORIES 



53 



where M is a normalization constant, and ^q(xq, vq) is the probability distribution 
from which we sampled xq and vq. The integral and it in the exponent are just 
notation, because they are not really well-defined (like 'Ricmann integrals' and 
'derivatives') as a consequence of the properties of the Wiener process. Again, 
we do not go into the full mathematical details here (for that, see [65 J. A more 
important quantity for the purpose of this thesis arises if we compare the path- 
probabilities of different dynamics to each other, by considering the action of the 
process with respect to a reference process. For this reference we define a new 
Langcvin dynamics: 

dxt = Vtdt 

mdv t = -mrfVtdt + V2VdB t (4.7) 

Note that it is the same as before, only we have taken the force F to be zero. We 
denote the path-probability measure of this process by dV°. Then we have that 



e 



- A{u) = ^(w) - exp | -L £ dt[2mv t + 2m 7 « t - F t (x t )]F t (x t )^ (4.8) 



This is the Girsanov- formula for our one-dimensional diffusion process. This 
quantity is well-defined, as the term in the exponent with v t , actually a notation 
for 



L 



j 1 n — 1 

dtv t F t (x t )= lim y> ti+1 - v ti )F u {x u ) (4.9) 

i=0 



is mathematically well-defined. However, because of the properties of the Wiener 
process we can't treat this 'integral' with the normal (Ricmann) rules of integration, 
and have entered the domain of stochastic integrals. The integral as defined in ()4.9[) 
is called a 'stochastic integral in the Ito interpretation,' or just an 'Ito integral.' We 
explain more on these stochastic integrals at the end of this section. We conclude 
for now by saying that, like in the previous chapters, we use this path-probability 
density to define expectation values for observables. If one defines the time-evolved 
probability density as fj,t(x,v) = (S(xt — x)S(vt — w))„ , then one can prove that 
its evolution equation is exactly the Fokker-Planck equation, see Appendix 151 
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4.4 Overdamped diffusions 



In many physical cases (e.g. molecular motors) the system is submerged in a highly 
viscous environment. This means that the friction coefficient 7 is very big. If on 
top of that the mass of the system is very small, one enters the overdamped region, 
in which the system after each random kick relaxes quickly to the local expected 
velocity, determined by the force F and the friction. Because of this, one is not so 
much interested in the changes of the velocity anymore, just in the positions. 

Mathematically one takes the limit of 7 — > 00 and m 0, while keeping \ = — 
constant and finite. This limit is not trivial [M] [Ml 1104] , but the result is that one 
can ignore the term mdv t in the Langevin equation (|4.2|) . If one then replaces in 
(|4.2p the term mjv t dt by mjdx t , one is left with the single equation 



with D = x 2 ^- We call this the overdamped Langevin equation. To distinguish 
the previous Langevin equation from this one, we call from now on (|4.2[) the 
under damped Langevin equation. The corresponding Fokkcr-Planck equation for 
probability distributions is 



which defines the probability current. This equation is in the literature often called 
the Smoluchowski equation, and the high-friction limit the Smoluchowski limit. In 
the special case that F(x) = — 9 U ^ , we can easily find the stationary distribution 
P- 



with P = ^. Just as in the underdamped case, this corresponds to equilibrium. 
Moreover, in this case the stationary probability current j p is actually zero. 

Let us also provide a simple example of a noncquilibrium dynamics: suppose the 
particle moves on a circle of length 1. We take the force F(x) = f a constant. 
This can't be written as the derivative of a potential. One can solve the stationary 
Fokkcr-Planck equation, though: the stationary distribution is just the uniform 
distribution p(x) = 1. The stationary current is j p = xf, which is a constant but 
is not zero. 




(4.10) 



ar + te** = o Wlth 



3^ t = xFm - D — 



(4.11) 



P ( X ) = ± e -m.) 
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4.5 Overdamped diffusions in more dimensions 



To be more general, let us work in a c?-dimensional space, and let xt be a d- 
dimcnsional vector, representing the position of a single particle. Then the most 
general set of d overdamped Langcvin equations, one for each component of xt, is 

BD 

dx\ = Y^bCij(xt)Fi(x t )dt + + \/2D(xt) ij dBi] (4.12) 

3 

In this equation F t still represents a force, D and \ have become d x d symmetric 
matrices both depending on the position Xt, and there are d Wiener processes 
dB\ . The term with the derivatives of D is a result of the fact that a non-constant 
diffusion coefficient gives an extra net force on the particle [11)1] . We also stress 
that we make no difference between indices that are placed as superscripts or as 
subscripts, i.e. x l = xi. To reduce notation, we schematically write these Langevin 
equations in a matrix-form: 

dx t = X (xt)F(x t )dt + V ■ D(x t )dt + y/2D(x t )dBt (4.13) 



In such a notation, X F is to be interpreted as the matrix X acting on the vector 
F. The corresponding Fokkcr-Planck equation is then 

^ i + V-j AIt =0 with j IH =xFtlH-DViH (4.14) 

The path-probability density is found in the same way as for underdamped 
diffusions: 



Mo(zo) 



| ~ jf dt[x t - x F t -V -D]- D-^xt - X F t - V • D] \ 



where a dot • denotes a scalar product of vectors. Again it is mathematically 
safer to look at the Radon-Nikodym derivative of this process with respect to a 
reference. Let us take as a reference ()4.13p with the force F put to zero. We 
denote the corresponding path-probability by V° and get 



e-^) = ^H = exp 

Mo 



j-^ dt[2x t -2V-D- X Ft}-D- 1 X F t ^ (4.15) 



This formula is the Girsanov- formula for overdamped diffusions where, like in 
the underdamped case, we still have stochastic integrals in our formula, meaning 
integrals which contain x. 
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4.6 Stochastic integrals 

In this section we shortly review some technical aspects of stochastic integration. 
For more information, see |4U1 158] . 

Ito integral Let us denote by q the configuration of the system. For underdamped 
diffusions this is (x, v), and for overdamped diffusions this is x. In (|4.9p and (|4.15[) 
one encounters integrals of the form J dtq k g t (qt), usually written as J dq k gt(qt), 
with g t some function of the configuration and q k an arbitrary component of q. 
This is actually just notation for the following: divide the interval [0,T] into n 
intervals of length At (so nAt = T) and define t{ = iAt for i = 0, . . . n, then for 
any function gt(q) 



Because of the special properties of Wiener processes, an integral defined as in 
(|4.16p can not be treated as a Ricmann integral. For example, for a Ricmann 
integral, it does not matter where the function g t is evaluated in each time interval, 
for stochastic integrals like (|4. 1 6[) it docs. From the way we defined the Langcvin 
equation it is natural that g t is evaluated at the beginning of each interval. A 
stochastic integral with this property is called an Ito integral. 

To compute what the integration rules are for Ito integrals, note the following: the 
increments of the Wiener process B t +&t—B t have a Gaussian distribution with zero 
mean and variance At. From this, one can prove that the quantity (B f +At — B t ) 2 
has a mean At and a variance of order (At) 2 . Therefore, one can safely write 



(B t+At -B t ) 2 = At+o(At). Analogously: fe fe i+1 -g|)(4 +1 -^) = 2D kj At+o(At), 



for any k, j = 1, . . . , d (Using V instead of D for underdamped diffusions). 

We compute here what the actual rules for integration are for Ito integrals. Take 
an arbitrary function g(q) not explicitly dependent on time. A Taylor-expansion 
gives: 




(4.16) 



c) 2 



g(q ti ) + o(At) 



dqidq k 
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« 9{lu ) + (Qu +1 -q ti )- Vg(g ti ) + (DV) • V<?(g tj ) At + o(At) 
As a consequence 



n-l 



/ dq t ■ Vg(qt) = lim Y7<?t s+1 - g ti ) ■ Vg t (g t J 

= lim V [g{q ti+1 ) - g (q ti ) - (DV) ■ Vg{q u )At + o(At)) 

n— >oo ^ — ' 



i=0 



7' 



ff(«r) - <?(9o) - / d*(I>V) • V 5 (g t ) 







The integral over time on the right-hand side is a normal Ricmann integral. 
Similarly one proves that for explicitly time-dependent functions gt(q) one gets 

j [dqfVgt(qt)+dt^(q t )}=g t (q T )-gt(q )- J dt(DV) -V '<?*(<?*) (4. 17) 



Stratonovitch integral Another stochastic integral that is useful is the Stratonovitch 
integral, denoted and defined by 

f dq k t o g t (q t ) = Inn - g *) gt ( g " + -^±i ) (4 .1 8 ) 

1,0 i=0 

A nice property of the Stratonovitch integral is that it is antisymmetric with 
respect to time reversal, if q only contains positions, not velocities (q = x). When 
velocities are involved one also has to reverse the signs of the velocities, and the 
Stratonovitch integral may not be time-antisymmetric in this case. One can easily 
prove that for any function g 

dq?og t (q t )= [ dq k t g t {q t )+{ dt(DV) k g t (q t ) (4.19) 
Jo Jo 

so that the Stratonovitch integral actually has the normal integration rules: 

j [dq t o Vg t {q t ) + dt^(qt)} = g t (q T ) - <?t(<Zo) (4.20) 
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Work From mechanics we know that work should be equal to the integral over 
a path of force times displacement. But in diffusion systems this definition does 
not completely specify work, as one has to choose which stochastic integral to use. 
However, we also know that the work should be antisymmetric under time-reversal. 
Because of this, it can be seen that the work performed by a force ft(x) during w 
is defined with a Stratonovitch integral: 

/ dx t of t (x t ) = J2 dx k t of*{x t ) (4.21) 
Jo k Jo 

where Xt denotes a position, not a velocity. 



4.7 Local detailed balance 

Let us see what restrictions the local detailed balance assumption (Section 12. 7p 
puts on our models. 



Overdamped diffusions Let us consider general overdamped diffusions (|4.13p . 
One can then compute the quantity S(u>) (as defined in (|2.12p ~) in the following 
way. The reference process, which is defined through (|4.13p but with F put to zero, 
is a pure diffusion, which is an equilibrium process with the uniform distribution 
as equilibrium distribution. Actually in unbounded state spaces this distribution 
is not normalizable, so technically we do not have an equilibrium process. In any 
case we still have the important property that 

dV° 



For the original process we can thus write 

S{w) = log g^) - log §|4 (a;) = MM - A (") 

which after a short computation, using (|4. 1 5[) and the definition of the Stratonovitch 
integral, becomes 

S(u)= [ dxtoD^ixtMx^Ftixt) 
Jo 



By the local detailed balance assumption this should be equal to the entropy flux 
into the environment. Let us consider the case where the environment consists of 
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one heat bath at inverse temperature /3. As an addition we write the force F t as 
the sum of a conservative part — VU with U representing the energy of the system, 
and a nonconservative part ft- Then we see that local detailed balance dictates 
that x = /3-D, because only in this case we have 



where in the brackets we have change of energy minus work done on the system 
(see 1)4. 2ip ). This relation between mobility and the diffusion coefficient is called 
the Einstein relation. It is commonly used in Langevin dynamics, but here we see 
it as a consequence of the local detailed balance assumption. 

Underdamped diffusions Let us return to the one-dimensional underdampcd 
case (|4.2|) . To compute S(us) we should take into account that the velocities 
change sign under time-reversal: 



We can apply the same argument as for over damped diffusions, and we see that 
local detailed balance dictates that T> = which is again called the Einstein 
relation. 




(4.22) 




Part II 

Fluctuations and Response 
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'Recent developments in non equilibrium statistical physics have convinced us 
that times are ripe for a review of the vast subject concerning the fluctuations of 
systems described by statistical mechanics. This issue is important even beyond 
the 'traditional' applications of statistical mechanics, e.g. in a wide range of 
disciplines ranging from the study of small biological systems to turbulence, from 
climate studies to granular media etc. Moreover, the improved resolution in real 
experiments and the computational capability reached in numerical simulations 
has led to an increased ability to unveil the detailed nature of fluctuations, posing 
new questions and challenges to the theorists." 

Umbcrto Marini Bcttolo Marconi, Andrea Puglisi, Lamberto Rondoni and 
Angclo Vulpiani, in their introduction to Fluctuation-Dissipation: Response 

Theory in Statistical Physics (2008), |76j 



Chapter 5 

Response 



In this part of the thesis, we investigate how a system responds to a perturbation, 
namely a small change in its energy. The central object that summarizes this is 
the response function. The goal is to relate this response function to correlation 
functions in the unperturbed process. In equilibrium systems this has already been 
done and the resulting relation is called the fluctuation- dissipation theorem. Out 
of equilibrium one needs to modify this relation. Extra terms appear that have to 
do with traffic, as defined in previous chapters. In this chapter we describe the 
general framework; in the next chapters more explicit models are examined. The 
research in this chapter was summarized earlier in JSJ'. 



5.1 Introduction 

The fluctuation-dissipation theorem is a standard chapter in statistical mechanics 
[221 E3 ES] . In Chapter 12.10.41 we already explained that a small change in an 
equilibrium system that changes the Hamiltonian Hq — > H = Hq — h V from 
time zero on, changes expectation values of observables Q(xt) at any time t. This 
change, up to linear order in h, is called the 'response' and is given by 

= p(Q{ Xt )[V{x t )-V(x )]}° p (5.1) 

h=0 

where the superscript h means that the average is taken in the perturbed 
system, and the superscript stands for the unperturbed system, which is here 
an equilibrium system. This equality (|5.ip is called the fluctuation-dissipation 
theorem. Actually it is usually stated in a different but equivalent form. In this 



~t- (Q(xt)) h p0 
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different form the parameter h is time-dependent: h = ht, where h t = for t < 0. 
The time-dependence of h t is something that is provided 'externally,' i.e. it is 
deterministic, it is controlled by the experimentalist. As an example, think of 
an Ising spin system, where the perturbation is a time-dependent magnetic field. 
The response is now the (functional) derivative of the expectation value of Q(xt) 
with respect to h s with s < t. The fluctuation-dissipation theorem for equilibrium 
systems then states that 

= P^-(Q(x t )V(x s ))° p0 (5.2) 

h=0 OS 

We will derive this form of the fluctuation-dissipation theorem as a special case of 
the more general formula later on. 

An early example of this theorem is present in Einstein's treatment of Brownian 
motion, where the diffusion constant, expressed as a velocity auto-correlation 
function, is found to be proportional to the mobility [35]. We will come back 
to this in the following chapters. Other famous examples include the Johnson- 
Nyquist formula for electronic white noise [551 El] and the Onsager reciprocity for 
linear response coefficients [85] [86] . The fluctuation-dissipation theorem is useful 
because it gives a relation between two quantities in essentially different processes. 
One can e.g. determine the response of a system without actually perturbing it. 

In this part of the thesis, we discuss and explain the results obtained in J2J[3][1]. 
This research investigates linear response in systems out of equilibrium. So far, 
approaches deriving a fluctuation-dissipation relation (FDR) for systems out of 
equilibrium have not found a physical unification and do not appear as textbook 
material. One reason may be that previous work has not been seen to identify a 
sufficiently general structure with a clear corresponding statistical thermodynamic 
interpretation. 

Aiming to provide a simple and general approach to FDR's, in [3j we have put 
forward a FDR for noncquilibrium regimes in a framework that may represent 
a unifying scheme for previous formulations. This is discussed in the remainder 
of this chapter. The application of this theory to Markov jump processes and 
overdamped diffusions was investigated in [J] , and the application to underdamped 
diffusions in [2J, and is explained in the next chapters. Additional work on 
fluctuation-dissipation relations for Markov jump processes was also reported in 
[75] , but we do not discuss this here. 
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5.2 Linear Response 

The unperturbed systems we consider are driven from equilibrium, but with a 
time-independent (time-homogeneous) dynamics. This dynamics can be far from 
equilibrium. We denote the energy of the system in a configuration x by U{x). As 
a perturbation the energy is changed by the addition of a potential: U —> U — h t V, 
where the time-dependent function h t is the amplitude of the perturbation. We 
assume that it is bounded: \h t \ < h < oo, with h a small number, and h t — 
for t < 0. We also assume that it is a continuous diffcrcntiablc function of 
time. Furthermore, if the configuration of the system includes velocities, we always 
assume that energies and potentials that we use are symmetric under the sign- 
reversal of those velocities. With the notation of Section (|2.5|) : e.g. V(x) = V(nx). 

We want to compute the influence of this perturbation on expectation values of 
observables Q(x). We restrict ourselves to a regime where a small perturbation 
only has small consequences, which excludes for example phase-transitions. This 
regime allows us to make an expansion of expectation values in orders of the 
parameter h, which we do up to the first, linear, order: 

(Q{x t ))^ = (Q(x t ))l o + [ dsh s R QV (t, s) + o{h) (5.3) 

Jo 

which defines the response function Rqv , which actually also depends on the initial 
distribution ^o'- 

(5.4) 

Applying the framework explained in Chapter |5] we use the action to rewrite 
expectation values of observables in the perturbed dynamics into expectation 
values in the unperturbed dynamics. Henceforth we denote by V h the path- 
probability density of the perturbed process and by V° that of the unperturbed 
one: 

<Q(**)>£ 

The averages are over all paths in the interval [0,t]. Because h t < h is small, the 
action is also small, and we can expand the exponential up to linear order in h: 

(Q(x t ))l = (<2(**)>° - (A{u)Q{x t ))l o +o{h) 



R Q v(t,s)= A-(Q( Xt )} h ^ 



Q{x t ) 
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Up to now this is only mathematics. Physics comes in when we split the action in 
its time-antisymmetric and time-symmetric parts (see Section 12.9ft : 

S ex (u>) = A(0u)-A(u) 

T ex {u) = A(0u) + A(u) 

where S ex (to) can, by the local detailed balance assumption, be interpreted as the 
excess entropy flux into the environment during the process to. Excess is meant as 
excess of the perturbed process with respect to the unperturbed process. 7^ x (w) 
is called the (excess) traffic. With this in mind we get 



(Q(x t ))l = (Q(x t )}[ 



- (S ex (uj)Q(xt))l B 



-o(/i) (5.5) 



We have thus written the change in the expectation value of Q as the sum of the 
correlation functions of Q with entropy and traffic. This is equivalent to writing 
that the response function (|5.4p equals 



1 S 

RQv(t,s) = - — (S , ex (w)Q(a; t ))° [ 



h=0 



(5.6) 



h=0 



This constitutes our most general but also our most vague result: the response 
function is the sum of two terms. One of the terms is expressed using the entropy 
flux into the environment, which is in principle known and measurable. The other 
term is up to here only mathematically defined and needs more investigation. 
Before we do that, we first discuss some general properties of the response function. 



5.3 General properties of the response function 
5.3.1 Relaxation 

We assume that the unperturbed dynamics is time-homogeneous. Then, as we 
assume in this thesis, the probability distribution of states relaxes to the stationary 
distribution, meaning that 

(Q(x t ))l ^ (Q(x t ))° p for i^cx) 

Suppose now that the perturbation we added to the system, is removed again after 
a time ti, i.e. ht = for t > t\. Then logically the system relaxes again to the 
stationary regime: 



(Qixt))^ -> (Q(x t ))° p for t -> oo 
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Using (|5.3p , this gives 

rti 

/ dsh s RQv(t, s) — > for i — > oo 
Jo 

As this is true for any /it, we can safely say that R,Qv(t,s) — > for t — » oo. 
However, throughout this text, we make a stronger assumption, namely that 

/oo 
dt|i?(M)l < oo (5.7) 

for any s. The reason for this assumption is that the excess heat dissipation 
remains finite, see Section f5. 4. 31 



5.3.2 Causality 

Physically, the response function should be zero whenever s > t. This is a 
consequence of causality, because the expectation value of an observable at one 
time can not be influenced by a perturbation at a later time. What we want 
to show here, is that causality is already embedded in our framework, and not 
something we have to put in afterwards by hand. In the discussion after (|2.4[) we 
argued that to take the expectation value of Q(xt), we only need to average over 
paths in the interval [0,i]. The perturbed dynamics that govern the probability 
of this path, and thus also the action A(ui), can logically only depend on h s with 
< s < t. So the perturbed expectation value of Q(xt) is independent of h s with 
s > t. This is indeed causality. In (|5.5p causality gives us that for s > t: 

(5.8) 

h=0 



h=0 



JJ^- (Tex(u)Q(xt))° 



/'() 



5.3.3 A constant perturbation 

For experiments and simulations, it is often more convenient to work with a 
constant perturbation. This means that h s = h for s > and is equal to zero 
for s < 0. In this case one will not directly measure the response function, but 
rather the time-integral of it. This is because (|5.3[1 reduces in this case to: 

{Q{xt))l - (QM)l +hf dsR QV (t, s) + o(h) 

Jo 
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and the response is then rather defined as 



h=0 



t 

dsR QV (t, s) 



This integrated response is sometimes called a generalized susceptibility, as a 
generalization of the susceptibility defined in thermodynamics. 



5.3.4 Response in the frequency domain 

Apart form a constant perturbation, a periodic perturbation is also often 
convenient in experimental settings, and response functions arc often measured 
in the frequency domain (using Fourier transforms). Let us consider therefore a 
perturbation with an amplitude h t = hocos(kt). We also restrict ourselves to the 
case that the unperturbed system before the perturbation was stationary: 

(Q(a*)>J = (<20)>° + ho f dscos(ks)R QV (t - s) + o(h ) (5.9) 

Jo 

Up to now we have always taken t — as the moment that the perturbation was 
turned on. If we change this time to t — — T, then it only changes the lower limit 
of the integral in (|5.9|) from to — T. We suppose that the perturbation was taken 
a long time ago, so we can effectively take — T — > — oo. Our assumption (|5.7|) 
ensures that this makes sense. The upper limit of the integral (|5.9|) can also be 
changed to +oo, as RQv{t — s) = for s > t due to causality. This naturally leads 
us to rewriting (|5.9[) in terms of Fourier transforms: 

/+oo p+co gifes _j_ g — iks 
ds cos(ks)RQv (t — s) = / ds RQy(t — s) 
-oo J -oo * 

= l -[R QV {k)e- m + R QV (~k)e lkt ] 

where the Fourier transform of the response function is defined as RQv(k) = 
dtR,Qy(t)e lkt . Using properties of Fourier transforms, and the fact that 
R,Qy(t) is a real- valued function, we arrive at 

(Q(xt)) h p - (Q(s)}J + h Q Re(R QV (k)e- lkt ) + o(h ) 

where Re(z) is the real part of a complex number z. So for these kinds of 
perturbations, what is actually measured is the Fourier transform of the response 
function. 
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Using complex analysis, one can prove |78j that R,Qy(k) is a continuous bounded 
function, as a consequence of the assumption (|5.7|) . 

5.4 The case of one heat bath 

The result (|5.5[) is a very general, but also a very vague result. To get more physical 
results we can of course investigate explicit examples, which we do in the next 
chapters. Before doing this however, we write down a more explicit form for the 
response function by utilizing the fact that entropy is a known physical quantity. 
To do this we restrict ourselves to systems in contact with an environment at a 
single temperature. This means the system can be driven away from equilibrium 
by a nonconservative force or by particle reservoirs at different chemical potentials. 
Throughout the rest of this chapter and the next, we keep this assumption. In the 
example of underdamped diffusions we will come back to the case of heat baths at 
different temperatures. 

Entropy, work and heat: In the case that the environment of the system 
is described by a single temperature, we can write the entropy flux into the 
environment as 

= PQ{u>) = 0[-AE + W(lj)} 

with Q the heat flow into the environment, AE the change of energy of the system, 
and W the work done on the system. The excess entropy flux is the extra entropy 
flux created by the addition of the potential —htV, and is thus equal to 

S ex (u) = PQ ex {u) = P[htY{x t ) - h V{x ) - f ds^V{x s )] (5.10) 

where the last term is the work due to the added potential —htV. We give a short 
argument to see why this is work: divide the time interval [0, t] into n segments of 
length At, and define U = iAt for i= 1, . . . , n. We then have that 

n 

h t V(x t )-h V(x ) = YfouVixtJ-fHt^VM] 

i=l 

n 

+YJ h ^ v ( x ^ - Vi^j] ( 5 - n ) 

i=l 

The left-hand side of this equation is a change of energy, and can thus be split into 
work and heat. In our discussion in Section 11.21 we defined heat as the work due 
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to (stochastic) forces we do not detect, and can not be externally controlled. If 
we apply this definition here, we can see that the second term on the right-hand 
side exists due to changes in the configuration, which are stochastic and can't be 
controlled. The first term is due to changes in the parameter h t , which is controlled 
externally. Therefore the first term in (|5.1ip is interpreted as work and the second 
as heat (see also [TH] )■ Taking the limit of n — > oo for the work-term, we get 
exactly the last term in (|5.10p . Note that we can't write the heat in terms of a 
partial derivative of V with respect to x, because that derivative usually does not 
exist, or even makes sense (e.g. for Markov jump processes). 



Response function With this knowledge we can rewrite the correlation of the 
entropy flux with the observable as follows: 

(S ex (co)Q(x t ))l o = P[h t (V(x t )Q(x t ))l o -h (Vtx )Q(x t ))l o 

ds^(V(x s )Q(x t ))°J 

= $ j\sh s ^(V(x s )Q(x t ))l Q (5.12) 

To write down the response function (|5.4p , it is convenient to define the functional 
derivative of the traffic: 



(5.13) 

h=0 



With this notation and using (|5.12p the response function (|5.4p becomes: 

R Q v(t, *) = (V(x s )Q(x t ))l ~ \ (r(w, s)Q(x t ))l (5-14) 

This is the most general formula for Markovian systems out of equilibrium, but 
in contact with only one heat bath at inverse temperature /3. We see that the 
first term on the right-hand side coincides formally with one half of the response 
function for equilibrium systems. It is of course the second term that is different for 
noncquilibrium systems with respect to equilibrium. For a constant perturbation 
ht = h, we get 

/" dsR QV (t, s) = ^ ([V(x t ) - V(x )]Q(x t ))l o -\f da (t(u>, S )Q(x t ))° 
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5.4.1 Causality and Stationarity 



For the case of one heat bath, we can combine (|5.14|) with (|5.8p to find for t > s: 



P§t (V(x t )Q(x s ))l = (rfatMx.))^ (5.15) 
We can use this causality relation to rewrite (|5.14|) for t > s: 



RQv[t ' S) = 2 \§~s W^M^lo - | (V(xt)Q(x s ))° 



1 r 

2 



[T(cj,t)Q(x s ))l o - (T(cj,s)Q(x t ))l o 



(5.16) 



Consider now the case that the system was in the stationary state when 
the perturbation was added, meaning (1q = p is the stationary distribution. 
Stationarity of the process means that it is time-translation invariant. As a 
consequence, the response function can be written as RQy(t,s) — R.Qv(t — s). 
Moreover, correlation functions computed in the stationary regime satisfy 



(V(x s )Q(x t )) p =(V(x )Q(x t - s )) p 
This means that in this case ()5.16p simplifies to 



R QV (t,s) 



P_d_ 
2ds 



(V(x s )Q(xt))° p + (V(x t )Q(x s )}° p 



1 



(r(u J ,t)Q(x s )) p -(T(u J: s)Q(x t )) p 



(5.17) 



valid for t > s. 



5.4.2 Equilibrium 

We prove here the fluctuation-dissipation relation for equilibrium systems. A 
system in equilibrium is necessarily in contact with only one heat bath, so (|5.14|) 
applies here. A system in equilibrium is also time-reversible. The time-reversal 
operator is defined as 8uj = (Trxt- s )o<s<t, where tt reverses the sign of velocities. 
Time-reversibility means that the expectation values of observables are equal to 
the expectation values of their time-reversed twins, e.g: 



(V(x s )Q(x t ))l - (V(x t )Q(nx s )) v p 
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where we used the assumption that V{x) = V(nx). From the definition of t{uj, s) 
(see (|5.13p ). it also follows that t(8uj, s) — t(oj, t — s), so that for t > s: 

(t(oj, s)Q(x t ))° p = (t(w, t - s)Q(ttx ))° p = (t(w, t)Q(irx s ))° p 

which by the causality relation (|5.15p equals 

(r(w, t)Q(nx s ))° p = p- (V(x t )Q(nx s ))° p = -p— (V(x )Q( Xt - s )}° p 
Substituting this equality in ()5. 16|) we arrive at 

R QV (t,s)=P^(V{x s )Q{xt))° p for t>s 

which is exactly the fluctuation-dissipation theorem in equilibrium. Note that for 
a constant perturbation ht = h the response becomes 

i 

d s R, QV (t, s)=P ([V{x t ) ~ V(x )]Q(x t ))° p 
as in (|5.ip . 

5.4.3 Dissipation 

In equilibrium it is known that the response function is closely related to the energy 
dissipation of the system into the environment |78| . This is already seen in the fact 
that the response function is then expressed through the fluctuation-dissipation 
theorem as a correlation functional between the observable and entropy flux. In 
systems driven out of equilibrium things get more complicated: even when the 
system is not perturbed there is already a heat dissipation. This is called the 
'housekeeping' heat that is needed to maintain the (unperturbed) noncquilibrium 
stationary state. Perturbing the system then gives additional heat. We see here 
that the prediction in |88j that the usual equilibrium relation between response 
and dissipation is preserved when taking into account only the excess heating and 
ignoring the housekeeping heat, is indeed true in the following sense. 

For a system in contact with only one heat bath, the entropy flux into the 
environment is equal to j3 times the heat dissipated into the environment. For 
each trajectory ui, this heat can be split in the heat of the unperturbed process 
and the excess heat of the perturbed process with respect to the unperturbed 
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process. In (|5.10p wc already wrote down explicitly the excess heat. We assume 
a perturbation of the form h t = hocos(kt), like we did in Section (|5.3.4|) . and 
compute the excess heat over one period of the perturbation [0,T = =£-] in the 
perturbed system: 

<Q«M>; = h T {V(x T )) h p -ho{V(x )) h p - j o dt^{V(x t )) h p 

= j\th t ^{V(x t )) h p 
We can write the expectation of V using the response function as in ()5.10p : 

(V(x t )) h p = (V(x))° p + ^ [R vv (k)e^ kt + R vv {-k)e m ] 

Substituting this, and the explicit expression of h t into the expression of the heat, 
gives us 

(Qex{uj)) h = ^ [ T dtcos(kt) [-R vv (k)e~ M + R vv (-ky kt ] +o(h%) 
z Jo 

= i J^L[R vv{ - k )-R vv{k )] +o{h l ) 

= nhllm[R vv (k)} + o(hl) 

We see that the excess dissipated heat can be expressed through the imaginary 
part of the Fourier transform of the response function. For systems that arc 
(unperturbed) in equilibrium, there is no housekeeping heat, so the excess heat 
is then total dissipated heat. Although the excess dissipated heat only specifics 
the imaginary part, the Kramcrs-Kronig relations in complex analysis connect the 
real and imaginary parts of complex functions to each other |78j . This means that 
when one of them is known, the other can be computed from it. 

Let us conclude here by noting that the imaginary part of the Fourier transform 
is equal to the Fourier transform of the time-antisymmetric part of the response 
function (|5.14|) : 

r(t-s) = l -[R vv {t,s)- R V v{s,t)] 

(rKi)l/( Is )>;-(r(,,^W);' 



- (3 ( V (x s )V(x t )) u p + - 
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which is the same as (|5 . 1 7[) for Q = V, but (|5.17p is only valid for t > s. For t > s, 
we have that r(t — s) = Rvv(t — s ), while for t < s the response function is zero. 
The antisymmetric part of the response function therefore contains all information 
of the whole response function. 



Chapter 6 



Markov jump processes and 
overdamped diffusions 

In this chapter the general framework outlined in the previous chapter is applied to 
the cases of Markov jump processes and overdamped diffusions. These two classes 
of models share a common property, simplifying the fluctuation- dissipation relation, 
as we shall see in \6. For a few explicit examples, simulations have been done 
to visualize the terms in the fluctuation- dissipation relations. The work explained 
here was written down in J^. 

6.1 An explicit formula 

In this chapter we only consider systems in contact with one heat bath, so that 
we can use formulae like ()5.14p and ()5.16p . The advantage of considering specific 
models is that we can get more explicit results. To do this, we need to compute the 
traffic, i.e. the time-symmetric part of the action for our models. More precisely, 
we need to compute the functional derivative of traffic, as defined in (|5.13p 

6.1.1 Overdamped diffusions 

Consider the case of general overdamped diffusions in d dimensions (|4.13[) . Adding 
the potential —h t V(xt) in Langcvin equations is straightforward. The perturbed 
Langcvin equation becomes (we arc still working in the Ito convention): 



dx t = x(xt)[F{x t ) + h t VV(xt)]dt + V • D(x t )dt + y/2D(x t )dB t 

75 



76 



MARKOV JUMP PROCESSES AND OVERDAMPED DIFFUSIONS 



Remember that the local detailed balance assumption here implies that x — fiD. 
In the same way as we derived (|4.15p , we can here derive the action that describes 
the relative probability of paths of the perturbed process with respect to the 
unperturbed process: 

dV h 



/ dx s h s WV + ^- f dshtVV ■ [V • D + X F + IhtxVV] 
Jo 1 Jo 1 



2 

The time-antisymmetric part of the action is given by a Stratonovitch integral: 



S ex (oj) = A{9cj) - A(uj) =P dx s o h s VV 

Jo 

as a consistency check, let us rewrite this using ()4.20|) : 

/ dx s oh s WV = h t V(x t )-h V(x )- / ds—^V(x s ) 
Jo Jo ds 

which is exactly (|5.10p . More importantly, we can compute explicitly the time- 
symmetric part of the action: 

r* i 

Jo 1 



so that the functional derivative with respect to h s becomes: 

= [V ■ (D(x,)V) + x{x.)F(x a )V\V{x a ) (6.1) 



t(0J, s) = ft —Tex{u) 



h=0 



and the fluctuation-dissipation relation: 

RQv(t,s) = ^(V(x s )Q(x t ))l o (6.2) 
-2 ([V • (D(x s )VV(x s )) + x(xs)F(x s )VV(x s )]Q(x t ))l o 
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6.1.2 Markov jump processes 

For Markov jump processes, it is more ambiguous to define what 'adding a 
potential' means. Starting from a general Markov jump process with transition 
rates k(x,y), the local detailed balance assumption asserts that (see (|3.10p ): 

k ( X >y) _ e -P[U(y)-U(x)-W(x,y)] 

k(y,x) 

where U is the energy of the system, and W(x, y) is the work needed for a jump x — > 
y. Adding a potential —htV changes the transition rates to new time-dependent 
rates k t {x, y) which, again by the local detailed balance assumption, have to satisfy 

k t( x >y) _ e -P[U(v)-U(x)-W(x,y)-htV(.v)+htV(x)] 

Still, this docs not completely specify the perturbation. There are many 
possibilities. The most common in literature however is the following one, which 
we use throughout this chapter: 

k t (x,y) = k{x,y)e^ [v{v) - v(x)] 

For the treatment of fluctuation-dissipation relations for more general perturba- 
tions, see [75] ■ From (|3.7|l we arrive straightforwardly at 

dV h 

= -§£Mnz s )-n* s -)] 

s<t 

+ f ds V k(x„ y)[e^ v (*,)-V(x a - )] _ i] 

The time-antisymmetric part of the action is here 
S ex {cu) = pJ2 h s [V(x s ) - V(x a -)} 
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one can check that this is consistent with (|5 . 1 Of) . The traffic is the time-symmetric 
part of the action: 

T ex (uj) = A(6uj) + A(u) = 2 f ds J2k(x s ,y)[e^ v ^- V ^ - 1] 

Jo 



and its functional derivative: 

pY,Kxs,y)[V(y)-V(x s )} (6.3) 



The fluctuation-dissipation relation then becomes 
R Q v(t,s) = ^{V(x s )Q(x t ))l 



-f ( iOMv) - V(x s )]Q(x t )\ (6.4) 



6.1.3 A general formula 

An important property of both Markov jump processes and overdamped diffusions, 
is that the functional derivative of traffic is a state function, see (|6.ip and ()6.3p . 
i.e. t(o;, s) = t(x s ). With this in mind we use the equality (|5.15[) : 

0^ (V(x s )Q(x t ))l - (r(x s )Q(x t ))l (6-5) 

which is true for any s > t, Q and //o- We use the (backward) generator of the 
unperturbed Markov process, which is defined (see Appendix [AJ as follows: there 
exists an operator L such that for any state function / and any initial distribution 
Mo: 

j t (f(xt))l = (Lf(x t ))l 

This L is called the backward generator of the (unperturbed) Markov process. If 
we use this generator and take Q = 1 in (|6.5|) then we get: 

( T ( X s))°a„ — P (LV(x s ))° 
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and because this is true for any s and any /xq, we actually have that 



t(x s ) = pLV(x s ) (6.6) 

Comparing (|6.ip and ()6.3p to (|A.8|) and (|A.11|) we can check that this is indeed 
the case. This expression also gives an interpretation of t{x). Using the definition 
of the generator L, we see that 



r(x) = PLV(x) = Pj t (V(x t ))° x \ t=o 

where the superscript x means that the initial configuration is fixed at x. Thus 
t(x) is the instantaneous expected change in the potential V when started at the 
state x. 

The fluctuation-dissipation relation becomes : 



R Q v(t, a) = |^ (V(x s )Q(x t ))l o ~ | (LV(x s )Q(x t ))l o (6.7) 

which is the most general formulation when restricted to Markov jump processes 
and over damped diffusions. These are large and useful classes of models. Moreover, 
notice that the result is true for any observable Q(xt), any potential V and any 
initial distribution /iq. The result is thus true in three noncquilibrium situations: 



1. The case in which there are nonconservative forces. 

2. The case in which there are different particle reservoirs at the same 
temperature but with different chemical potentials. 

3. The case in which the system has an equilibrium dynamics, but has not yet 
relaxed to equilibrium. 



Of course combinations of these nonequilibrium conditions are also possible. In 
this sense the result (|6.7|) is both very general, and explicit in terms of the generator 
of the process. The downside is that experimentally, this generator is not always 
known. On the other hand, in the next section we show some examples where this 
formula leads to (in principle) measurable results. 



6.2 Examples 



In this section we give three physical examples to clarify the structure of the 
fluctuation-dissipation relations. For two of these examples, simulations have been 
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made by Marco Baicsi, to provide some visualization and a check of the results 
[¥]. For simulations it is often more convenient to use a constant perturbation: 
h s = h, s > 0. As discussed in Section 15. '6. '6 1 this gives an integrated version of the 
fluctuation-dissipation relation. For this section, it is convenient to introduce the 
following notation: 

• The generalized susceptibility: 

If extended to t f oo, gives the change in noncquilibrium stationary 
expectation when adding a small potential. 

• The correlation function, originally coming from the cntropic term in (|6.7p : 

C(t) = {Q{x t )V{x t ))l Q - (Q(x t )V(x ))l o 

• The term coming from the traffic term in (|6.7[) . (extra with respect to 
equilibrium) : 

K{t) = - f ds (LV(x s )Q(x t ))° 



Jo 

representing an integrated correlation function. 
• The average of C and K: 

C NE (t)= 1 -[C(t)+K(t)] 

The fluctuation-dissipation relation is in this notation expressed by 

X(t) = PC NE {t) (6.8) 

6.2.1 Driven Kawasaki dynamics 

As a first example we consider an exclusion process as a model of ionic transport 
through a narrow channel. The model we use is very similar to the exclusion 
process described as example 2 in Section 13.81 differing only by the addition of an 
interaction potential. We repeat quickly: the model is described by a collection of 
n sites, labelled by i — 1, . . . , n, each holding cither one particle (x l = 1) or none 
(x l = 0). A configuration x is thus an array of ones and zeros. In the bulk of this 
system no particles are created or annihilated, only jumping to neighbouring sites 
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is allowed (Kawasaki dynamics) . At the edges i = 1 , n particles can move in or out 
from reservoirs with density d\ and d n , respectively To this a nearest neighbour 
interaction is added, with energy U(x) = — X)"=i x l x l+l . Moreover, we can add 
an "electric" field E promoting particle jumps to the right. 

We have to construct transition rates for particles hopping to neighbouring sites 
and rates for creation and annihilation at the edges. For example, consider the case 
that a particle enters into site i = 1 from the reservoir. Denote the configuration 
before that jump x, and after the jump y, where y = x except that x 1 — while 
y = 1. Using the local detailed balance assumption, we know that 

k ^v) = e ^i e -§iu(y)-u(x)] = d i c -?\u( v )-u( x )] 
k(y,x) l-di 

This does not specify the rates fully of course. For simulational purposes we 
therefore define the rate for a particle entering at i — 1 as 

k(x,y) = d!ip(x,y) expi^-~[U(y) - £7(as)] j , ip(x,y) = ip(y,x) 

where for the moment ip{x,y) is an arbitrary function. Then of course, a particle 
leaving gives 

k(y, x) = (l- di)1>{x, y) exp \j[U{y) - U(x)} 

and similar for particles entering and leaving at site n. In the bulk of the system 
particles can hop, e.g. to the right. Let x and y denote configurations which differ 
by the fact that a particle at site i has hopped to the right: x % = l,x z+1 = and 
y l = 0, y' l+1 = 1. Then local detailed balance gives 

^| = exp{-/3[£%) - U(x) - E]} 

Where E is an electric field promoting hops to the right. We therefore take the 
hopping rates to be: 

k(x,y) = tp(x,y) exp - U(x)- E]\ , ip{x,y) = ip(y,x) 

and similarly for jumps to the left. For simulational purposes we have specified 
4>{x, y) by an additional condition, namely that k(x, y) + k(y, x) ~ 1 for all x, y. 



The system is driven from equilibrium by: 
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i) setting different reservoir densities d\ ^ d ni or 

ii) setting a nonzero electric field E > in the bulk. 

We choose the total number of particles Af(t) = X)"=i x% as observable. We also 
introduce a perturbation V(s) equal to A/"(s). This particular perturbation is 
equivalent to changing the chemical potential of both reservoirs with a common 
shift: fa — > fj.i + h: 

Transition rates for the perturbed process are thus multiplied by a factor e^ h / 2 if 
a particle enters the system, and by e~^ h ^ 2 when a particle leaves; transitions in 
the bulk are left unchanged. 

In this case the traffic term is 

LM(x) = J2 Hx, y) [A%) - N{x)\ = J{x) 
y 

which is a current. This current represents the expected change of Af per unit 
time, i.e., it is the rate of change in the number of particles from the two possible 
transitions at the boundary sites. Thus, for x — > y the transition modifying x 1 , 
and for x — > z the transition modifying x n , we have 

J(x) = [A%) - M{x)]k{x, y) + [N{z) - Af(x)]k(x, z) 

We have numerically verified that x, C, K as defined earlier, are all equal to each 
other under equilibrium conditions. While C = K to excellent precision, the shape 
of x depends weakly on h, and is found to converge to C only for h sufficiently 
small. In fact, one can pretend exact matching only in the limit h — > 0, but 
h = 0.01 turns out to be sufficiently small to achieve a good convergence. 

A representative example for the noncquilibrium case i) is shown in Fig. IG.lf a). 
One can see that each of the functions C(t) and K{t) is a poor approximation of 
the response, while the agreement between Cjvis(i) and x(t)/P is excellent. 

In Fig. 16.1( b) we show an example of noncquilibrium condition ii). Again, only 
CjvbW matches Curiously, a comparison of this example with the 

previous one reveals that C can be cither larger or smaller than K , even for two 
noncquilibrium conditions that look pretty similar, in the sense that they yield a 
current in the same direction for a relatively simple system. 
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Figure 6.1: Plot of the quantities involved in Eq. (|6.8p . for (a) case i) (E = 0) with 
n = 10, j3 = 1, h = —0.01, and reservoir density unbalance d\ = 0.9, d n = 0.1, 
and (b) for case ii) (di = d n = 0.5) with n = 10, ft = 1, h = -0.01, E = 3. 




Figure 6.2: Response and fluctuations of the over damped particle in a tilted 
periodic potential, as discussed in the text, with inverse temperature /? = 0.2, 
mobility v = 1 and perturbation h = 0.02. (a) Steady state regime, with initial 
distribution equal to the stationary one, and force / = 0.9. (b) Transient regime, 
with initial position x = and force / = 0. 
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6.2.2 Diffusion on the circle 

Consider the over damped Langcvin equation (|4.10|) for a particle position x t 6 S 1 
(on a circle) with a force F(x) — f — U'{x). Here the prime denotes differentiation 
with respect to space. The constant force / drives the particle around the circle 
and out of equilibrium. To this equation a perturbation is added: 



we renamed the mobility v to avoid confusion with the integrated response x(t). 
The diffusion constant D and the mobility v are related by the Einstein relation 
v = f3D, with /3 the inverse temperature (local detailed balance). 

This example has been recently experimentally realized as reported in |45j . for 
testing the results of [THJ HH], see also [HI E2]- Moreover, its noncquilibrium 
stationary distribution p is known analytically, see |71| . 

Here we show the result of a simple simulation of the overdamped particle studied 
in the experiment, with a potential V(x) = sin(ar), and also U(x) = Q(x) = V(x), 
and a constant force /. The functional derivative of the traffic t(x) = j3LV(x) 
of a particle at position x can be computed by applying the generator of the 
overdamped dynamics 



to the potential V(x), which gives t(x) — j3[f — cos(x)] cos(a;) — sin(a;). We have 
measured the correlations C(t) and K(t) as well as responses \{t) for small h. 

Fig. I6.2f a) shows that for strong stationary noncquilibrium / 3> there is a large 
difference between the integrated correlation function of the traffic and that of 
the entropy production. However, their average Cjve(^) agrees very well with the 
response x(t), as it should. It is also important to remember that our approach 
works for non-stationary regimes as well. In Fig. 16.2( b) we show an example of a 
particle starting at time t = from a non-stationary initial distribution fi(x) = 6 x ,o 
(i.e. its position is xq = 0), but for the rest not forced outside equilibrium, / = 0, 
to emphasize the transient character of this situation. Again, we can see that the 
response is well estimated by Cjvjs(t)- 

In the next chapter we treat the incrtial version of this model. 




(6.9) 
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6.2.3 A generalized Einstein relation 

Consider the ovcrdampcd Langcvin equation in R d (|4.13|) but with constant v 



dx\ = v ij [F j (x t ) + h8 lk ]dt + V2D 3 dB: 



(6.10) 



with repeated indices j = l,...d summed over. The F includes all forces and h 
is the small constant perturbation. Note that this means that the perturbation 
potential is V(x) = x k . With such a potential the fluctuation-dissipation relation 
connects mobility and diffusion which are actually defined as follows. The true 
mobility M (in contrast to v) is defined as the response 



M ik = lim — (x\) 
t-j-oo dh x ' 



= lim - [ 4r (x\) 



(6.11) 



h=0 



while the real diffusion matrix (in contrast to D) is given by 



1 r 

V lk = lim — ds 

2t J jo 



dr ((xl-(x*) p )(x k ~(x k ) p )) 



As (|6.1ip is a response function, we can easily compute it within our framework. 
Note that LV{x) = u kj F j (x): 



h=0 



We rewrite this response function by using the following equality, derived from 
taking the average of the Langevin equation ( ([6.1UP with h = 0): 

because the average of dB t gives zero. Using this equality we get 



1^1 



h=0 



2 jo 



£ / n ds[(± k ±i);-(x k s p (xiy p ] 

Cds(Fi(x s )(xl-(x l ) p ))° p 
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Now all that is left is integrating this relation over t, dividing by t and taking the 
limit for t — > oo to arrive at 

M lk = pV lk - lim - [ ds [ dr ( F j (x s )(xt - (x l )°))° (6.12) 

2 t^°o t J J \ p I p 

Observe that in equilibrium (F derives from a potential) , the second term in 1)6.121) 
vanishes because the observable Xt is anti-symmetric, and (i)^ q = 0. Then, the 
equilibrium fluctuation-dissipation relation holds with M = f3T). Moreover, when 
F = 0, (pure diffusion) these quantities can be explicitly calculated, and are found 
to be equal to M lk = u lk ,V lk = D lk . We see that for pure diffusion the fluctuation- 
dissipation theorem is equivalent to the Einstein relation. But even for equilibrium 
systems, when F =/= the fluctuation-dissipation theorem is not equivalent with 
the Einstein relation, although there is still a simple relation between mobility and 
diffusion. 

Out of equilibrium, this simple relation is violated, as there is a correction, namely 
the second term in ()6.12p ). Note that this is completely compatible with the 
condition v = f3D, because that is actually a relation for the reservoir, which 
is in thermal equilibrium at inverse temperature (3. Similar results for a one- 
dimensional overdamped diffusion were reported before in [Sj [57] . 



6.3 Connection with effective temperature 

A reason, not mentioned before, that the fluctuation-dissipation theorem in 
equilibrium is so useful and powerful is that it specifies the temperature as a 
universal parameter: regardless of what potential we use and what observable, 
the relation between response and correlation function is always governed by that 
same single parameter. In this sense it is not strange to wonder what remains of 
this parameter out of equilibrium. 

Therefore, even though the fluctuation-dissipation theorem is violated out of 
equilibrium, one attempts to restore it by the introduction of an effective 
temperature T c ^, in the sense 

Many studies have been devoted to the study of this prefactor, in what sense it 
perhaps resembles a thermodynamic temperature-like quantity for some classes 
of observables and over some timescales; we refer to [T^l [TTJ [3TJ [SO] for an entry 
into the extensive literature. Clearly, whatever the purpose of the discussion, an 
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exact expression of the response should help, especially when entirely in terms 
of explicit correlation functions. The first calculations in this sense are in [Tl"] 
and they have been referred to as the "no field-method" [HJ- In particular, for 
purposes of simulation or numerical verification of (|6.13p we do no longer need to 
perform the perturbation by hand. In fact, now we can write the ratio T/T cff = X 
entirely in terms of correlation functions 

X = X QV (u;t,s) = -\l-\ — - — (6.14) 

with numerator and denominator in (j6. 14[) each having a specific physical meaning 
in terms of entropy and traffic. An effective temperature is obtained as the ratio 
between the traffic and the entropic term: if for some observables (V, Q) and over 
some time-scales, 

Y±{V(s)Q(t))l = {LV(s)Q(t))l 

for some Y, then X = (1 — Y)/2. Equilibrium has X = 1 = —Y. In the case 
where LV w as for a conserved quantity, then Y = and T e ^ = 2T. Finally, X 
and the effective temperature T en ° get negative when the traffic term overwhelms 
the entropic contribution. 

One should understand that ()6. 13|) represents a rather optimistic scenario. 
Formula (|6.13[> mimics ()5.2|) by replacing a function depending a priori on the 
observable, the potential, the initial distribution and on other parameters as 
temperature and time, by just one parameter. Why should there be also out- 
of-cquilibrium a single parameter and a useful notion of temperature in its 
usual thermodynamic understanding? Moreover, how would it depend on the 
observables V and Ql (See [77] for a very recent discussion.) Answers to these 
questions have been partially given but arc often restricted within a context of 
mean field systems or for small fluctuations, effectively dealing with calculations 
as in Section 16.3.21 similar to calculations for scalar fields as in [2T] and in [T3] . 
In fact, the optimism in ()6.13p is a sort of conservatism as it tries to return 
to equilibrium-like formulae. We take a different attitude: the violation of the 
equilibrium fluctuation-dissipation relation (FDR) is an opportunity to discover 
new connections between response and the relevant newly emerging physical 
quantities as traffic. 

6.3.1 Explicit calculations for pure diffusions 

In simple overdamped diffusion equations, one can explicitly calculate the 
correlation functions, and see when the concept of effective temperature could 



88 



MARKOV JUMP PROCESSES AND OVERDAMPED DIFFUSIONS 



make sense. Similar calculations have been done in |12U21j . The simplest example 
is Brownian motion in one dimension: 



dx t = h t dt + V2DdB t 

where we have taken the potential V(x) equal to the position x. The generator 
of the unperturbed dynamics is (see Appendix E| L = DA, so that LV = and 
rpcS _ 2J 1 c f. Virasoro's example in [2T]. In that last reference, what are called 
"flat directions" can be associated to perturbations with LV = 0. 



6.3.2 Explicit calculations for linear diffusions 

Next, consider the following model for diffusions with a linear force (harmonic 
potential). Fix parameters a, B £ R, D > and look at the linear Langevin 
dynamics for a global order parameter M G R, 



dM(t) = -a M(t)dt + h t Bdt + V2D dB(t) 



The ht,t > 0, is a small time-dependent field. The generator of such a dynamics 
dynamics (on observables /) is (see AppendixEJ L h f(M) = [-a M+h t B] f'(M) + 
Df"(M), using a prime to denote differentiation with respect to M. Such a 
dynamics can arise as a Gaussian approximation to a rclaxational dynamics of 
the scalar magnetization M (no conservation laws and no spatial structure) valid 
in high enough dimensions (above d = 4 for the standard Ising model). Then, 
in a way, a = corresponds to the critical (massless) dynamics and a > 
is a paramagnetic dynamics (high temperature), see |12j . By taking D \. we 
exclude the diffusive aspects and we can think then of gradient relaxation in the 
low temperature regime. 

The equilibrium (reversible stationary density on R for perturbation B = 0) is 



with zero mean and variance (M 2 ) = D/a. We now start from an initial 
distribution If A*o 7^ P then the system is not in equilibrium. We can compute 
the response function by calculating the expectation value of M at time t in the 
perturbed dynamics. We do that by deriving the following differential equation 





i(M(t))l = {L h M{t))t 



a{M{t))t +h t B 
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the solution of which is 



Jo 



^ s mt))l\ h = = Be-"^ (6.15) 

which docs in fact not depend on (Af(0)} (and thus also equals the equihbrium 
result). In the same way, we use for s < t: 

±(M(t)M(s))l = (M(s)L°M(t))l = -a{M{t)M{s))l o 
to arrive at 

2f„\\0 -at 



(M(t)M(s))l = (M\s))le 



Using same strategy we can calculate the expectation of M 2 (s) . Denoting M5 2 
(M 2 (0)) , the correlat ion function for < s < t is 



(M{s) M(t))l o = M 2 e ~ Q (*+ s ) + ^[ e -«(*- s ) - e -«(*+-)] 
and hence 

^-{M (s) M(t))l o = -aMfe- a < t+s) + D[e- ai - t - s ^ + e - Q ( t+s )] 

If = P hi the last expression (thus replacing Mg by D/a) we find 

^{M{s)M{t))° = De~ a{t - S) 
as 1 

which, in comparison with (|6.15p specifics the equilibrium inverse temperature to 
be equal to /3 = B/D. 

The traffic term is obtained from LM = —aM, and thus 

(LM(s) M{t))l = -aM§e- a{t+s) - D[ e - a{t - s) - e - a{t+s) ] 
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Clearly, 

A-(M(t))* o (h = 0) = ^{±(M( s )M(t))l - (LM( S )M(t))l} 
as it should. 

For the issue of effective temperature we compute the ratio Y as 
-a(M(s)M(t))l o _ D — all* - De 2as 



Y = Y(M ;s,t) 



£<M(s)M(*)>° D-aA-q + De^ 



Remark that Y is independent of i, as long as t > s. In that notation, the effective 
inverse temperature is T e ^ = 2T/(1 — Y). In equilibrium Y = — 1 while Y = 1 for 
D = and Mo ^ 0. If a = 0, then Y = as in the pure diffusion case. For a > 0, 
if we let s — > do while keeping t — s = u fixed, we get 

lim Y(M ;s,s + u) = -1 (6.16) 

sf+oo 

so that T e ^ — T. This case is referred to as the paramagnetic case, while T e ^ = 2T 
for a = is the critical quench, cf. [T^] . 



6.3.3 Effective traffic 

Let us try to sec how the notion of effective temperature could be seen as a one- 
parameter reduction of a general equilibrium-like FDR that is valid also outside 
equilibrium but with an effective dynamics. The starting point is comparing the 
equilibrium formula (|5.2p with the more general non-equilibrium one (|6.7p . By 
this we see that in equilibrium (|5.2p is equivalent with 

RQv(t,s) = -p((LV)(s)Q(t)) p (6.17) 

In other words, the fluctuation-dissipation theorem in equilibrium can also be 
called a fluctuation-traffic theorem; the two terms on the right-hand side of (|6.7p 
are simply the same in equilibrium. Therefore, for the purpose of getting closer 
to equilibrium response formulae one really has the choice to mimic cither (|5.2p 
or rather (|6.17p . The first leads to the ambition of effective temperature ()6. 13|) . 
the latter to the new notion of effective traffic. But the latter is also much richer. 
In fact, a combination of (|6.7p with (|A.6[) shows that the exact noncquilibrium 
response formula (|6.7p can indeed be written in the equilibrium form (|G . 1 "[) : 



R% v {t,s) = -(G fla V{s)Q(t)}l 



(6.18) 
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with a new effective traffic 

G^V = ^-[L\(mV)-VL^ + (mLV] (6.19) 

Here is the forward generator, defined by (see Appendix 1X1 (|A.2[) ): 

dxg(x)Lf(x) = / dxf(x)L^g(x) Vf,g 



where the integration has to be replaced by a sum for Markov jump processes. The 
operator G M acting on V in (|6.19|) has the following exact property: it is itself a 
generator but of a new dynamics for which /i is an equilibrium distribution (i.e. 

(/(G><?))° = (3(G M /)>° fOT a11 f,9, see (El))- Thu s, in (EUl) the generator 
G Ms is the instantaneous equilibrium generator with respect to the time-evolved 
distribution p a . 

We can rewrite in terms of the adjoint generator (see (|A.4[0 . which is defined 
using the stationary distribution of the process: 



dxp(x)g{x)L* f(x) = J dxp(x)f(x)Lg(x) V/,5 
With this we get 

GnV = ^[L*(^V)-VL*(^) + ^LV\ (6.20) 
2/i p P P 

In the stationary noncquilibrium case, we have p s = p and (|6.19p is 

G P =§(£ + £*) (6.21) 

replacing L in (|6.17|) . For equilibrium dynamics L = L* (sec Appendix lA"]) . and 
(|6.18|) reduces to ()6. 17|) . In stationary noncquilibrium, if the perturbation V 
is 'time-direction independent' in the precise sense that LV = L*V, then the 
noncquilibrium response reduces to the equilibrium formula (|6.17p and X = 1. 
See [52] for very related conjectures and observations. 



Chapter 7 

Underdamped diffusions 



In this chapter we treat the case of underdamped diffusions, for which the formula 
\6. 7\) is no longer correct. We can, however, still calculate the response function 
and express it in terms of correlation functions of observable quantities. For several 
explicit examples the response is calculated and simulated. This chapter describes 
work reported in [J)/. 



7.1 Model 

In this chapter we consider an extension of the one-dimensional underdamped 
diffusions explained in Section 14.11 There we considered the dynamics of just one 
particle. Now we consider k particles in d dimensions. This means that the state 
of the system is now given by (x,v) = (x\, X2, ■ ■ ■ , x n ; Vi, V2, ■ ■ ■ , v n ) G R 2 ™ of 
positions and momenta, with n = kd. These particles are subject to mechanical 
forces: a potential U(x) taking care of the coupling and pinning of the positions 
(the pinning is also thought to confine the positions to some finite volume) , and a 
nonconservative forcing fi(x). Each particle is connected to its own heat bath at 
an inverse temperature /3j. This gives the following set of 2n Langevin equations: 

dx\ = v\ dt 

dU , 

rmdvl = [fi{x) - g^fa) " matvfidt + v^A dBj(t) (7.1) 

Again, we make no distinction between upper or lower indices: x% = x l . To avoid 
confusion, we always use k to denote the components of the vectors, and s,t 
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to denote times. For the simple undcrdampcd diffusions in (|4.f [) . we found by 
the local detailed balance assumption the Einstein relation between diffusion and 
friction coefficient. Let us investigate what it gives here. For this we write down 
the path-probability density for paths uj = (xt,Vt)o<t<T- In the same way as 
explained in Section 14.31 we find, formally: 



= Jf CXP \ ~ ? 4~k J dt ^ m ^ ~ F ^ + m i"/i v i\ 2 \ ( 7 -2) 



with Fi = fi — and N a normalization factor. The entropy flux into the 
environment is then 

Oil x t ,vt® Jo 

Note that the quantity 

m i v l - ti = -mriiVi + 2Di— — 

at 

is the (stochastic) force on particle i originating from the z-th heat bath. The 
integral over time of this force times the velocity of the particle thus gives the heat 
flux from the i-th reservoir to the system. As a consequence S(oj) is the entropy 
flux into the environment if and only if Di = m ^ 7 ' for all i. Note that the system is 
driven out of equilibrium by the nonconservative force, but also by the difference 
of the temperatures of the heat baths Pi. 

At time zero, the probability density of the system being in state (x, v) is denoted 
as always by no(x, v). The Fokker-Planck equation determining the time-evolution 
of this probability in this case is 

j t lH + V ■ J Mt = (7.4) 

for V = (V x , Vu) and for the probability current J M = (J^, J^) with 

dU da 
(J*)t = rriiVifi, (J")i = (fi - HViU- A ^ (7.5) 



7.2 Perturbation 

The perturbation is a potential V(x) added to the unperturbed Hamiltonian 

2 

H a = J2i + U(x) — > H a — h s V(x) with small time-dependent amplitude 
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h s ,s > 0. Wc compare the path-probabilities of the perturbed versus the 
unperturbed process, using the action. By (|7.2|) we find 

= e- A ^ = CX p\ Sex{ui) ~™ 

where the action is split in its time-antisymmetric and symmetric parts. The 
time-antisymmetric part is: 

Se*(w)=Y>/ h s — (x s )vlds (7.6) 

Note the following: if all temperatures are the same, /3, = j3, then £ ea ;(w) = 
(3 J h s ^-(x s )ds, which is consistent with (|5.10|) . The time-symmetric part of the 
action gives: 

^ rt gy r qjj 

T{i0) = J h s — (x s ) {fl(Xs) ~ T^(Xs)}ds - ITlidvl + o(fl) 

(7.7) 

The last stochastic integral can be interpreted in the Ito or Stratonovitch sense, 
this does not matter in this case, see (|4.19p . However, the stochastic integral is the 
reason that we can't write the functional derivative of traffic as a state function. 
Indeed, dVi contains information about two consecutive states, not one. 



7.3 Result 

The model that we use in this chapter is in contact with more than one heat 
bath, possibly at different temperatures. Therefore we can't use the formulae 
derived in Section 15.41 Wc therefore fall back on the more general formula (|5.6D . 
We consider general observables Q(xt,vt) which we simply denote by Q(t). The 
immediate application of (|7.6|) and (|7.7|) to (|4.19|) gives 

i—i 
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However, because of the singular nature of the white noise, v is not a good 
observable. We therefore rewrite that part of the correlation function. We take 
the stochastic integral in the Ito sense, which is easiest here. This means that we 
have to interpret the correlation function containing ii as follows: 



<}Q(t) 



We rewrite this inspired by the product rule for derivatives to 



JTo7 ( [ £ (Xs+£K+e ~£; (a;sK]0(<) 



/'() 



1 / dV dV \ 



Finally taking the limit, we obtain a useful form: 



dV , . A d ,dV 



— ( Xa )vlQ(t)) = Q(t)) M (7.8) 

Mo 



dxjdx l 

3 J 



With this the response function becomes 

siT m)h ^ = 2E ft fe (a ^ Q( ^ (7 ' 9) 

Z— 1 2 

1 v-^ 1 ,dV , . . „ , . 9/7 , ^ . 

-2 E^few [/*(*.) ow>«, 

1 d y-^rrii dV 

-2E^W s,) ^ ,0(t)>w 



This may be a complicated formula, but we now have expressed the response 
function in terms of correlation functions of observables which are in principle 
measurable. 
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7.4 Examples 

The result (|7.9p is a complicated formula. This is why we have investigated it in 
several physically interesting examples. For two of these examples, simulations 
have been provided by Marco Baiesi [2J. 

We remind that a regime out of equilibrium can be created in (|7.ip by letting 
the inverse temperatures fti differ from one another. Another way of going out 
of equilibrium is to introduce nonconservative forces {fi}, like external fields that 
are rotational. A final possibility that we can consider is to start from an initial 
condition that is not stationary, and thus to observe the response in a transient 
regime. All of these possibilities are covered in the following examples. As in the 
previous chapter, numerical results are better presented with integrated responses. 
We use slightly different definitions than in Section 16.21 

• The integrated response, also called generalized susceptibility: 

* w =/W(m)=^ W( <: W( < 

Jo h 

• The correlation with entropy: 

C{t) = I (S(w)Q(% 

• The correlation with traffic: 

• The average 

C(t) + K(t) 
^ne(tj = 2 

Contrary to the examples in the last chapter, here we have embedded /3i's in the 
definitions of the correlation functions. The integrated response relation is thus 
x(t) = Cne(i)- We choose for simplicity particles with mass equal to one in all the 
examples. 

7.4.1 Langevin particle in a periodic potential 

Recently there have been experiments testing the response of an overdamped 
particle (high viscosity limit) in a periodic potential [IS]. In the previous chapter, 
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section 16.2.21 we have discussed simulations of that system. Here we look for 
the changes in an undcrdampcd set-up, allowing e.g. for the particle to have a 
considerable mass and to obey a noisy Hamiltonian dynamics. 

We consider here a system consisting of one particle in one dimension: its position 
is Xt € 5" 1 (on a circle) and the velocity is v t E R. Then, the equations (|7. 1 [) 
simplify to 

dxt = vtdt 

dv t = F(t)dt - ~fv t dt -h t g(t)dt + V2D dB t (7.10) 
where we abbreviate 

F(t) = f — -^-(xt) (deterministic force) 
dx 

git) = — r-(xt) (perturbing force) 
dx 

The nonconscrvative force / is the driving and is taken constant over the circle, 
thus effectively tilting the conservative potential. This drives the system out of 
equilibrium. As there is only one particle, there is only one heat bath. 

At time s = the unperturbed system is taken to be in the stationary 
nonequilibrium p corresponding to (|7.4|) . and then for s > a constant small 
h is turned on. Hence, the integrated correlations are 

C(t) = =-pf ds {v s9 (s)Q(t))° p = p [{V(t)Q(t))l - <F(0)Q(*))°J 
Jo 

K(t) = =±y*ds{F(s)g(s)Q(t)) p - j\dv(s)g(s)Q(t))° p 

(7.11) 

Note that we left dv s in the definition of K(t). This can of course be rewritten 
as explained in Section [7.31 However, in simulations one discretizes time, and dv s 
is known at every time-step. It can therefore be used as an observable here. We 
take V(x) = U(x) = cos(27rcc) like in the previous chapter, and again also Q = U. 

In Fig. l7.ll wc visualize the various terms of the fluctuation-dissipation relation, for 
three scenarios with different viscosity, increasing from left to right. The response 
is well reproduced by Che(£), even if we perform a numerical integration with 
dt = 10~ 3 . Oscillations in the response are visible for small viscosity; at higher 
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Figure 7.1: Integrated correlation functions of the entropic term C(i), of the traffic 
term K(t), and their average CneW gi ym g the response in noncquilibrium, and 
the integrated response x(t) calculated directly with h = 0.01. Panels are for 
simulations with various friction coefficients: 7 = 0.2 (left), 7=1 (center), and 
7 = 5 (right). Other parameters: T=l/0 = 0.2, / = 0.9. 




Figure 7.2: As in figure 17771 but with fixed 7 = 0.2 and varying force /, from left 
to right: f — (equilibrium), / = 0.3, and / — 0.6 (case / = 0.9 is in the previous 
figure). 
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friction there is a monotonous drift towards a new stationary state (right panel in 
Fig. 17. ip . The incrtial regime is less sensitive to the perturbation, as x(t) displays 
only a small wiggling: a high entropy production is almost compensated by a high 
traffic. For the high viscosity regime, with this setting (/ < 1) the traffic term 
is close to zero compared with the entropic term, and their combination yields 
Che ~ C/2. 

In Fig. l7.2l we can follow the response as a function of the driving /, with / = for 
equilibrium. For / ^ the entropic term C(t) can be quite different from Chc(£)- 
In equilibrium all the terms coincide, as expected. 

7.4.2 A generalized Einstein relation 

Let us consider the underdamped version of the example in the last chapter, 
Section 16.2.31 For this we take the general Markov dynamics (|7.ip . with the 
following simplifications: we take V{x) = Xj for an arbitrary j, and for the 
observable we take Q = Vk for a k which can be different from j. We also take 
the field ht = h to be constant. In this case the response function is related to the 
mobility of the system, i.e. the way in which the average velocity changes under a 
constant added force. We fix a large time u and define the time- averaged mobility 
by: 




starting from /xo at time zero. We wish to connect this to the velocity fluctuations 
in the unperturbed (but driven) system: 

1 1 f u f u 

We try to be more general here than in the last chapter: we do not take the limit 
m j +oo. In equilibrium, when (1q is the Maxwell-distribution and with / and 
U equal to zero and all the temperatures are equal = /3), one can compute 
that Xjk = ^-5j,k and Djk = -jpp$j,k- Indeed, also in the underdamped case, 
the Einstein relation coincides with the fluctuation-dissipation relation for pure 
diffusion. For general equilibrium systems we still have = /3T>jk- This is no 
longer true out of equilibrium. With (|7.9|) we can give the explicit modification. 
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We substitute V = Xj and Q = Vk into the integrated version of (|7.9|) for constant 
h to obtain 

-5kjf*<"*(-'-S(*'W) M 

2Dj 7 ds x J * mo 

The integrand of the full right-hand side is zero for s > t due to causality, so we 
can as well integrate s from up to u > t. Integrating then t gives us 

1 f u f u I dU \ 

+ ^£ dt ¥ -*•*)„ 

This relation reduces to the familiar Einstein relation if the unperturbed system 
is in equilibrium. Furthermore very formally when the over damped limit is taken, 
i.e. we neglect changes in momenta, then the last term in the relation drops and 
we recover the relation found in (|6.12p . 




7.4.3 Coupled oscillators 

We now consider coupled one-dimensional oscillators at different temperatures T$ 
in the stationary or in a transient regime. This means that each index i represents 
a particle which can move in one dimension. In (|7.ip we then set a conservative 
potential U that is the sum U = Y^i=o v( x i+i ~ x i) °f local couplings between the 
oscillators i and z + with tp(x) = ^x 2 + jx 4 . Boundary conditions are imposed by 
keeping xq = = 0. For simplicity we take all Di = D. A basic perturbation is 
given by switching on an external field on the j-th particle V(x) = —Exj. Taking 
the velocity Q = Vk at site k to be the observable, the variable excess in entropy 
flux (|7.6p reduces to 

/ 

S e x(u) = -fijE J ds v 3 s h s 
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while the traffic equals 



7» 



E 



J 



ds v{ + ——(x s ) h s 



D 



o 



From (|7.9p we have 




p h=o 



PjE(v{v k t ) 



o E d 
p ~ 2Dd^ 




2D 



E 




(7.12) 



This last relation is still valid for all times s, t and is automatically equal to zero 
for s > t (causality) . Remembering that /3j = = 1 /Tj we can rearrange formula 
(|7.12|) for the situation where s < t (like we did in (|5.16p ): 



Note that the right-hand side now shows a formal space-time symmetry for 
exchanging j k, s -o- t. In equilibrium the symmetry is true on spatial level 
alone, j <H> k, because time-symmetry is automatic. That is then an instance of 
Onsagcr reciprocity [SSj [55] . 

Choosing constant h = 0.03, in Fig. 17.3( a) we show the response in a system 
with ?i = 11 oscillators, with a linear gradient of temperature Tj = i/10 = 
perturbation applied on site j = 1 and response tested at central site k = 6. In 
Fig. I7.3f b) instead we have constant temperatures Tj = 0.2, but we start from a 
state out of equilibrium, by choosing pi(0) = 10 and the other momenta equal to 
zero. 
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Figure 7.3: Visualization of the fluctuation-response relation x(t) = Cnc(i), for 
n = 11 coupled oscillators, with parameters, D = 0.01, /i = 0.03, and (a) Tj = «/10, 
(b) Tj = 0.2 and transient regime as described in the text. 



Chapter 8 

Conclusions and literature 



To conclude this part of the thesis we give an overview of previous formulations 
of nonequilibrium fluctuation dissipation relations, and present our conclusions of 
our own work. 



8.1 Overview of previous formulations 

The literature on (extensions of) the fluctuation-dissipation theorem is vast, and we 
can't list all possible and even essential contributions. The equilibrium formulation 
spans all of the previous century, while nonequilibrium versions started to appear 
since the 1970's and very much continue up to now. Early works include pi I1U2] 
and also [231 HZ], where we see a discussion within the theory of stochastic 
dynamics. In contrast to |47l 148] our unperturbed process is time-homogeneous. 
Coming to more recent times, violations of the equilibrium relation have been 
most often discussed in transient regimes (equilibrium dynamics but not relaxed 
to equilibrium). For example, in the context of ageing phenomena |17| 151] . 
much thought has been given to making sense of an effective temperature as was 
discussed in Section 16.31 

However, recently much new work has been also directed to find generic extensions 
of the fluctuation-dissipation theorem to nonequilibrium steady states and to 
discussions of the dissipative elements in relaxations to nonequilibrium. There 
is for example the line starting from Sasa et al jlHJ [501 [EU HE] which treats 
nonequilibrium heat effects. In particular [15] might also be useful in real 
experiments because the response function is there directly connected with the 
energy dissipation, which we also discussed in Section 15.4.31 We do not know yet 
how to relate that to the new ideas surrounding the traffic term in our work. 
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For other recent extensions of the FDR, we refer to [HI [HI SOI IH3 [HE] ■ We have 
also mentioned in [5J [Jj (but not in this thesis) how our approach is related to the 
co-moving frame interpretation of Chetrite, Gawgdzki et al |T3J |TBJ S3 . There 
the equilibrium fluctuation-dissipation relation is recovered for systems out-of- 
cquilibrium by going to a description in a co-moving frame. One disadvantage of 
this approach, and of the approaches in the previous references, is that one keeps 
the stationary density p (or its logarithm) as an observable in the correlation 
functions. In our approach the largely unknown distribution only enters in the 
statistical averaging. On the other hand, the other approaches come with a new 
and still interesting interpretation (e.g. response in the time-reversed dynamics in 
[B] and co- moving frame in |15l 116] ) . 

A first generalized fluctuation-dissipation relation giving a response formula in 
our sense appears in Section 2 of [21] (Kurchan et al). It treats an overdamped 
Langevin dynamics for a variable y(t), used for soft spin models, and the resulting 
equation (2.10) in [5T] is exactly our formula (|5.16|) for Q = V equal to y, where 
(|6.6p can be used. Another treatment for systems of Ising spins is done in [53] IM] 
(Lippicllo et al), there the response is also investigated to higher orders in h. 
A more general treatment for jump processes is offered in [26] (Diczcmann), in 
particular its equations (16)-(17), see also |7S]. In contrast, we have emphasized 
the interpretation via entropy and traffic in nonequilibrium fluctuation theory 
(more is to come on the meaning of traffic in the next part of this thesis). In fact, 
that interpretation is exactly what makes a systematic generalization possible at 
all. The only study in which we recognize some of the ideas related to the traffic 
term is in [23] (Ruche), in the context of dynamical systems. 

8.2 Conclusions 

We have studied linear response relations under general nonequilibrium conditions 
(stationary and not). From physical constraints on the probability of trajectories, 
we have obtained in Chapter [5] a general fluctuation-dissipation relation for the 
response of a driven system to the addition of a potential. Most generally in 
formula (|5.6p . and more specifically for systems in contact with an environment 
at one specific temperature in (|5.14|) . Our scheme lifts the nonequilibrium FDR 
beyond formal first order perturbation theory applied to a specific dynamics, by 
identifying in general physical terms the statistical quantities that determine the 
response: entropy and traffic. 

Chapter [5] has dealt with jump processes and with overdamped diffusions, which 
is the usual set-up for discussions on the violation of the fluctuation-dissipation 
theorem and for the possible emergence of an effective temperature. The general 
fluctuation-dissipation relation for these classes of models is (|6.7|) . As in this 
relation the correlations with entropy and with traffic are expressed in terms of 
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explicit averages, they constitute a formula ready to use in a general context. For 
example, estimates of these correlations can be obtained with usual averaging in 
simulations, without any need to know further details or approximations of the 
stationary density of states. On the theoretical side, several previous approaches 
can be recovered or extended within the our scheme. In many cases these results 
have been discussed in specific model dynamics or for specific observables. It is 
interesting that there is a unifying approach with statistical interpretation behind 
this very broad variety of previous results. 

In Chapter we have applied our scheme to under damped diffusions, resulting in 
(|7.9p . It is seen that the correlation with entropy involves dissipation over different 
reservoirs, each one with its own equilibrium temperature. The correlation with 
traffic is again written in terms of observable quantities. 

Generally, we like to stress the emergence of traffic, as an important player out 
of equilibrium, complementary to the entropy flux. In equilibrium entropy and 
traffic give the same contribution to the response function. Out of equilibrium, 
they detach and the response needs to be evaluated in terms of correlation 
functions with both entropy and traffic. So far traffic is in general not understood 
operationally for real experiments, but in the previous chapters it has been 
expressed as a statistical average and correlation function of theoretically known 
forces. The fluctuation-dissipation relations we derived also give observational 
significance to the notion of traffic, which so far has mostly appeared as a 
theoretical concept in fluctuation theories (see also the next part of this thesis). 

In the ongoing research it will be interesting to consider more general and other 
models, e.g even non-Markovian systems. Also within the models we considered, 
more general types of perturbation are possible and interesting. For example, 
what will happen if the perturbation is not of the potential type but actually 
changes the nonequilibrium part of the dynamics? Think of changing one of the 
temperatures of several heat baths, or adding an extra nonconscrvative forcing. 
In all these cases our results will have to be modified, of course, but we believe 
that the general method and framework we use is applicable to this wider set of 
questions. Specifically, the concepts of entropy and traffic will remain major actors 
in nonequilibrium systems. 



Part III 

Dynamical fluctuations 
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'Large deviations estimates have proved to be the crucial tool required to handle 
many questions in statistics, engineering, statistical mechanics, and applied 
probability." 



Amir Dembo and Ofer Zeitouni, in their preface to Large Deviations Techniques 

and Applications (1998), [H] 



Chapter 9 

Large deviations in statistical 
mechanics 

In 1910, Einstein proposed to invert the equation on Boltzmann's gravestone, to 
express probabilities of fluctuations in terms of entropy: 

W = exp{S/k B } 

Such a formula shows that the probability of a fluctuation (a deviation of the 
macrostates from their equilibrium values ) is exponentially small and the entropy 
is the function that governs this exponential behaviour (rate function). This idea 
is generalized in the application of large deviation theory to statistical mechanics. 
This theory mathematically deals with the exponential decay of probabilities of large 
deviations in stochastic processes, as a correction to the law of large numbers. It is 
very useful and forms a natural mathematical formalism for statistical mechanics, 
because thermodynamic potentials as entropy and free energy naturally emerge as 
rate functions. This chapter forms a brief introduction for the next chapters. 

9.1 Introductory example: a discrete ideal gas 

Consider a discrete ideal gas [37]. We mean by this a system consisting of N 
independent particles, labelled by i = 1, . . . , N. The energy uji of each particle is 
a stochastic variable, drawn from the set {Ek\k = 1, . . . , to}. The probability that 
particle i has an energy cjj = Ek is pk, where p = (pi, . . . , p m ) is a probability 
vector, i.e. ^2 k pk = 1. The total state of the system is given by lu = (oji, . . . , lon)- 
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Because the particles are assumed to be independent, the probability of the whole 
state w is then 



N 



Prob(w) = JJProb(w<) = [ 



i=i ik=i 



E k Pk 



As a first step towards a more macroscopic description of such a system, we are not 
interested in the exact energy of each particle, but rather in how many particles 
have a certain energy. We define therefore the fraction of particles that have an 
energy E k : 



1 N 



(9.1) 



where S is the Kroneckcr delta. We call this fraction the empirical occupation of 
Ek- The empirical occupation vector for the collection of energies is 

Vu = [Ml), ■ ■ ■ ,M m )] 

Note that this (macrostate) is a stochastic variable, and at the same time defines 
a probability distribution, because X)fcP^(^) = 1- 



The law of large numbers By the law of large numbers, we expect this empirical 
occupation vector to resemble p = (pi, . . . , p m ) when the particle number N is very 
large. More precisely, we define the space of probability vectors 

m 

A" = { M = ... , Mm ) £ [0, in ]T/i fc = 1} 

fc=l 

The collection of possible empirical occupation vectors for N particles form a 
subspace of X ', which we denote by Ajy. The metric we consider on X is 

d(fx, v) = ^(pk - Vk) 2 ^l/Gft 

k 

Then the law of large numbers dictates that for any positive eel 
lim Prob(d(p w , p) < e) = 1 

N— yoo 

However, for finite N, fluctuations/deviations from this expected behaviour are 
always possible. If we want to know more about these fluctuations, we have to go 
beyond the law of large numbers. 
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Central limit theorem If wc want to go beyond the law of large numbers, we 
could use the central limit theorem. Formally, the central limit theorem then says 
that in a distributional sense 

Prob ^V^ ~ z) -> J-expf-V -1} (9.2) 



for z £ R m and Af a normalization factor. To derive (|9.2p is not trivial and we will 
not elaborate on this, because it is not very important for our purposes. We do 
stress that for large N this theorem only says something about the probabilities of 
events for which p u — p is of the order of 1 /V~N). We call these small fluctuations. 
To know more, wc have to consider the probabilities of large fluctuations. 



Beyond the central limit theorem In a simple system as this, it is not difficult 
to compute the probabilities of arbitrary fluctuations. Let us therefore take an 
arbitrary p G A at. This p can be totally different from p and is therefore called a 
(large) fluctuation. Elementary combinatorics give us 

TV' 

Prob(p " = Af) = (Np 1 y...... i Np m y.^ ■■■■■ p - 

For systems with large N this expression as such is not very convenient. Wc 
therefore use Stirling's approximation for large N to derive the following: 

1 m f \ 

- log Prob(p w = p) « - ^ log ( y ) (9-3) 

k=l ^ Pk ' 

The approximation becomes exact in the limit of N — > oo. For this reason, wc 
write the probabilities in the following form: 



Prob(p w = p) sa e 



-NiQi) 



where I(p) = X^feLi ^ ^ s mm us the relative entropy between the probability 

vectors p and p. One can easily prove that I(p) > in general. Moreover, it is 
only zero for p = p. 

Even though we have made an approximation, this result still gives more 
information than the law of large numbers and even more than the central limit 
theorem: it tells us that the probability that the empirical occupation vector 
to be equal to some p ^ p is exponentially small in the number of particles 
(only when p = p there is no exponential decay, in accordance with the law of 
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large numbers). At least, this is the dominant behaviour for large N. It is not 
very surprising that there is an exponential behaviour in the number of particles, 
because all particles are independent. However, also in many systems which do 
have interactions, this exponential behaviour is observed. This may not come as 
a surprise if we realize that many interactions are short-ranged, and parts of the 
system that are sufficiently far apart are still approximately independent. 

Such an exponential behaviour as in (|9.3p is an instance of a large deviation 
principle. As it turns out large deviation theory forms a natural framework 
in which statistical mechanics can be embedded. It incorporates in a natural 
way many key features, as the framework of thermodynamic potentials and the 
variational principles that characterize equilibrium. 

In this chapter we introduce the reader into some aspects of the theory of large 
deviations, by using the example introduced above. The introduction that follows 
here is based on [5711100] , and for more explanation we therefore refer to those texts. 
For a thorough treatment of the mathematical theory, we refer to [23] , while for the 
embedding of the theory of statistical mechanics in the large deviation formalism 
we refer to [551 1551 I5T1 EDO] . 



9.2 The large deviation principle 

In the example above, we defined the empirical occupation vector p u , and 
computed approximately its probability distribution for large N. Let us try to 
be mathematically more precise. We consider the probability that the empirical 
occupation vector is close to some given probability vector /i£l. We thus take 
an arbitrary (small) positive e € R and consider the probability P(d(p UJ , p) < e). 
Writing Bn(h, e) = {v e Ajy|d(/U, v) < e}, this probability becomes 

P(d(p u ,fi) <e)= P(P^ = ") 

Although it is not trivial, one can prove [57] that this probability satisfies a large 
deviation principle, stated here in a simplified way: 

1 m / \ 

- lim - log P(d(pu, fj) < e) = VVlog ( — ) + 0(e) = I(fx) + 0(e) 

N^ooN ^ \ Pk J 

As this can be done for arbitrary small e, one often denotes this by 

- lim llogP(p u = A0=J(A0 (9.4) 

N— >oo IV 
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The functional / is here the relative entropy between the probability vectors \i 
and p. Generally in large deviation theory, / is called the rate function, as it 
quantifies the rate of exponential decay, or fluctuation functional., as it quantifies 
the probability of a fluctuation (deviation from the typical value) . 

To define the large deviation principle in general, one needs to go into many 
mathematical details [53]. This is not our goal. For our purposes, a definition as 
in (|9.4[) is enough. Moreover, we do not concern ourselves in this text with proving 
the large deviation principle. We do assume that, for the systems considered in 
the next chapters, those proofs can be done (in a physically uninteresting way). 
Instead we usually just assume the large deviation principle to hold, and refer to 
e.g. [22] [57] for full mathematical proofs. 

Following [3S1 11UU] , we denote 

to express a large deviation principle as in (|9.4p . 

We state here some properties of rate functions, without giving proofs, (for proofs 
we refer again to [1001 l^i] ): 

• For some p, € X, the limit in (|9.4p may not exist, because the probability 
density decays to zero faster than exponentially (supcr-cxponcntially) in N. 
In that case the rate function is set to +oo. 

• In some cases I(p) = for some /i, meaning that the probability density 
decays slower than exponentially (sub-exponentially). In the example, this 
only happens when p = p. In the systems we consider in the next chapters, 
we assume that the rate function only has one global minimum where / is 
zero. Actually this is equivalent with the assumption that there is a unique 
stationary state of the system. 

• Rate functions are positive: I(p) > 0. If they were not, the corresponding 
probabilities would diverge instead of decay. 

9.3 The contraction principle 

Suppose we are given a new random variable, defined through a continuous 
function of the old one: q u ~ h(jp u ). As we will see in the following, such a 
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function can be many-to-one. A heuristic argument gives the rate function for q 
in terms of the rate function for p. Through the large deviation principle we write 

p(q^ = I/) * f dfie- NI ^ 

Laplace's saddle-point approximation gives an exact expression for the following 
limit: 

- lim -i log / dpie- NIM = inf 7(/x) 

N^ca N Jn:h(n)=v n:h(n)=v 

One concludes that 

p(quj = v)x cxp{-iV inf I(p)} 

defining the rate function for q u as I q {v) = inf /i: fc( /1 )=i/ 1(h)- This way of computing 
one rate function as the infimum of another is called a contraction. 

Let us consider the following case. In absence of information about our system, we 
assume that the probabilities of all the energy levels of the particles arc the same, 
i.e. pk = 1/m. The rate function I(y) then becomes I(y) = logm + ^ fc fik log^fc. 
Up to some constants this is minus the Shannon entropy of the probability vector 
(i. Consider the probability that the total energy per particle (average energy) is 
equal to a value E, giving the following contraction: 

1(E) = inf I(n) 

We can find such an infimum by using Eulcr-Lagrangc equations with Lagrange 
multipliers to ensure the restrictions. We get k equations of the form: 

d 

0= 75— [logm + VMfclogMfc + /?(VMfc£fc -£) + 7 (V^fc - 1)] 
d ^ k k k 

where 7 makes sure that fi sums to one and (3 makes sure that the average energy 
is E. The solution of these equations is then a p of the form 

where Z is a normalization constant, and /3 is determined by ^2 k [ikEk = E. This 
is the same form as the probability distribution of the canonical ensemble! It arises 
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here as the most probable probability vector given an average energy E. The rate 
function becomes 

1(E) = log m - 0E - log Z 

One can check that H = p. If we return to the idea that entropy and probability 
are related by S — ks log-P, then the rate function / should represent minus the 
entropy per particle, divided by fcs- As a result dS/N = ksfidE. Indeed, the 
Lagrange multiplier /3 is nothing but the inverse temperature associated to this 
system. We also see that —4 log Z is the free energy of the system. 

In this example, we clearly sec that the ensemble theory of equilibrium statistical 
mechanics naturally appears in large deviation theory, at least for this example. 
The entropy plays the role of rate function, and the canonical ensemble is found 
through a contraction principle as the most probable probability vector given an 
average energy E. 



9.4 The Gartner-Ellis theorem 

Let us take an arbitrary vector M £ M. m . We denote the scalar product between 
such a vector and the probability vector [i by M ■ fi — y, fe MkHk- The scaled 
cumulant generating function A(M) for our example is defined by 

A(M)= lim ±-log(e NM - p ") 

According to the Gartner-Ellis theorem [MHU], if A(M) exists and is differentiablc 
for all M, then P(p^ = jj,) satisfies a large deviation principle with a rate function 
given by 

1(H) = sup [M ■ fi - X(M)} 

Al£R m 

This defines the rate function as the Legendre-Fenchel transform of A. The relation 
can also be inverted: X(M) = sup„ e ^[M-/j,— /(/j,)]. We will not prove this theorem 
here, but heuristically, one can argue that 

{e NM -P") x J dfjie"^- 1 ^ 

Similar as before, the saddle-point approximation of Laplace tells us that 
/ e NM- P „\ _ cxpjjv sup [ M • n - 1(h)}} 

H<£X 
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The Gartner-Ellis theorem can help for explicit calculations of rate functions, as it 
is sometimes easier to calculate the generating function A. Vice versa, sometimes it 
is easier to compute a rate function to get information about a generating function. 

As an illustrative example, we assume again that pk = 1/m. Consider the following 
generating function: 

\(E k )= lim ilog/e-^S*^) 

As we already know the rate function I{p) in this case, we can compute the 
generating function by 

X(E k ) = sup[-/3 E ^k - /(/*)] 
" k 

We find that the canonical distribution p,^ = ^e~^ Ek solves this variational 
problem. As a consequence, the generating function \(E/.) = logZ. Remember 
that we interpret the rate function I(p) as the entropy per particle S/(NkB), and 
— 4 log 2 is the free energy F. The Gartner-Ellis theorem thus relates these two 
by the Legendre transform, schematically: 

F = sup[(£) M - TS(j*)] 

9.5 Typical behaviour and small fluctuations 

The large deviation theory incorporates the law of large numbers and the central 
limit theorem in the following sense. 

Law of large numbers Suppose that the rate function I{p) has a unique global 
minimum at p = p. First of all, we can show that this minimum should satisfy 
I(p) = 0. This follows from the fact that A(0) = 0: 

= A(0) = sup[-/( M )] = -I(p) 
Secondly we see that for any (measurable) set B <Z X: 

This tells us that P (p u £ B) — > whenever p ^ B, and P{pu, £ B) — > 1 whenever 
p £ B. This is the law of large numbers, with p the typical value to which p u 
converges. 
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Central limit theorem Given again a rate function I(p) with a unique global 
minimum at p. Suppose that the 1 rate 1 function is twice diffcrcntiable at p. We 
define 'small fluctuations' as values p close enough to p so that we can approximate 
the rate function by 



IE 



d 2 i 



2 j-j* dp, k dpi 



(pk - Pk)(pi - Pi) 



In this approximation, the probability density has the form of a Gaussian for large 
N: 



vt \ d 2 I 



2 j-f dp k dpi 



{Pk - Pk)(pi - Pi)} 



Which is (a form of) the central limit theorem. For this reason fluctuations in 
this regime are also called small fluctuations, Gaussian fluctuations or normal 
fluctuations. 

For our specific example, where I{p) = J2k 1°§ IT we have that 



d 2 I 



dpkdpi 



Pk 



so that for small fluctuations 

N^(p k - Pk) 2 



i v x — ^ 

P{Pw = p) ~ cxp{-— 2^ 



2 — Pk 



} 



which coincides with (|9.2p as expected. 



9.6 Out of equilibrium 

Because large deviation theory gives such a natural framework for equilibrium 
statistical mechanics, it is a natural candidate to give an extension to noncquilib- 
rium. As we said in the introduction, stochastic processes are an important tool 
in noncquilibrium statistical mechanics. 

The (mathematical) large deviation theory for Markov processes was thoroughly 
founded by Donsker and Varadhan in [571 123 HS1 ED]- In the context of physics, 
Onsager and Machlup [57] already used the large deviation approach in the 1950's 
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to examine relaxation to equilibrium in linear diffusion systems. A more general 
treatment of this problem, with a different emphasis, was done by Freidlin and 
Wentzell gTJ ITD3] 

In more recent years advances have been made in what is called static fluctuation 
theory [5j[5]. This is set in noncquilibrium stationary dynamics, and one imagines 
the system to be initially in the stationary regime a long time ago (at time — oo, 
the distribution of states was the stationary distribution). The probability is 
then considered that the distribution at time zero is equal to some other given 
distribution (a fluctuation). 

In this thesis we discuss dynamical fluctuation theory, in which time-integrated ob- 
servables are considered, with time as the large parameter. This is mathematically 
not a very big step: it boils down to renaming the particle number N to the time 
T. For example: consider the vector to = (u)\, . . . , Wjv) from the example above, 
ft describes the energies of the different particles. We could also see it as the 
consecutive energies of one particle through time, so that to becomes a trajectory 
like we have used throughout this text. The empirical occupation vector p u then 
gives us the fractions of time that the particle has spent in each energy state. 
The only mathematical difference is that we work with continuous time. This 
corresponds to measurements that are made continuously. We clarify this in the 
next chapters. Apart from having a different interpretation, this switch to time 
as a large parameter also allows us to consider small systems, without having to 
make some hydrodynamic approximations. 

In [701 EJ EU [M| we have reported the application of dynamical fluctuation theory 
on various classes of models. The major goal in this line of research is similar 
to that of fluctuation-dissipation relations: to find out what thermodynamic 
quantities govern the rate functions of the large deviations. Again, entropy and 
traffic turn out to be the basic ingredients for constructing this out-of-equilibrium 
theory 

We restrict ourselves in the next two chapters to the work reported in [70] [7T] in 
which respectively Markov jump processes and over damped diffusions are treated. 
However, the overall strategy in all models we have considered is the same: one 
starts by correctly defining the observablcs (the time integrated occupations and 
currents, as is explained there). Then, the rate functions (fluctuation functionate) 
are computed for the joint probability of occupations and currents, which can 
be done explicitly. The rate functions for occupations or currents alone can 
then be computed by the contraction principle, which is unfortunately not always 
possible. Throughout this calculation the physical aspect is most important: we 
keep track of the influence of entropy and traffic on the rate functions. We also 
show that the large deviation theory provides a way of defining thermodynamic 
potentials, although these potentials do not directly correspond to the ones defined 
in equilibrium thermodynamics. When only small deviations from the stationary 
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behaviour are considered, the rate functions are approximated to a quadratic 
dependence on the fluctuations (the regime where the central limit theorem 
applies). When the dynamics is also close to equilibrium, known variational 
principles such as the minimum entropy production principle are recovered. 



Chapter 10 

Driven overdamped diffusions 



We apply the theory of large deviations to time integrated observables in general 
overdamped diffusion processes. These observables are empirical occupation 
densities and empirical currents. Central is the joint rate function for occupations 
and currents and the role of entropy and traffic in it. This chapter represents work 
reported in \7Vj . 

10.1 Model 

We quickly recapitulate the equations describing overdamped diffusion, which we 
discussed in Section 14.51 

10.1.1 Overdamped diffusions 

General overdamped diffusions in d dimensions are governed by the Langcvin 
equation (|4.f 2p . interpreted in the fto way: 

dx\ = J2[Xio(xt)F j {x t )dt + -^f(xt)dt + ^2D(x t ) l] dBl] (10.1) 

3 

where x l denotes a spatial component of the position x. In the shorter 
vector/matrix notation this gives: 

dx t = x(xt)[f(x t ) ~ VU(x t )]dt + V • D(x t )dt + y/2D{xt)dB t (10.2) 
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In this equation we have explicitly written the energy U of the system, giving a 
conservative forcing, while ft represents a nonconservative force (i.e. the driving). 
Furthermore D and x are dxd symmetric matrices both depending on the position 
Xt , and there are d Wiener processes dB\ . We restrict ourselves to the case that 
the system is in contact with one heat bath at a temperature (3. Remember that 
the local detailed balance condition implies x = PD. 

We restrict ourselves to two types of boundary conditions: 

(1) periodic — the particle moves on the unit torus [0, l) d and the fields U, /, and 
X are smooth functions on the torus; 

(2) decay at infinity — the potential U grows fast enough at infinity so that the 
particle is essentially confined to a bounded region, i.e., the density and its 
derivative vanish at infinity. 

Under either of the above boundary conditions we can simply ignore boundary 
terms when performing integrations by parts. The particles arc essentially confined 
in their configuration space. 

The probability density [it evolves according to the Fokker-Planck equation (|4.14p : 



The stationary condition reads V ■ j p = 0. 

This class of models is still quite large. It is illustrative to consider a simple 
example of these dynamics: a particle moving on a circle (one dimension). The 
advantage here is that we can explicitly calculate the stationary distribution. 

10.1.2 Diffusion on a circle 

We consider a particle undergoing an ovcrdamped motion on the circle with unit 
length: 



The prime as superscript is a shorthand notation for the spatial derivative. 

The corresponding Fokker-Planck equation for the time-dependent probability 
density [it is 




3n = XV (F 



VU) - DVii 



(10.3) 



dx t = x{x t ) [f{x t ) - U'(x t )] dt + D'(x t )dt + y/2D(x t )dB t 



(10.4) 



dt 



i„ = xM/ - u') - zv 



(10.5) 



ENTROPY AND TRAFFIC 



125 



The stationary density p solves the stationary equation j' = 0, i.e., 

XP (f - U') - Dp 1 = j p (10.6) 
is a constant. For / = 0, equation (|10.6|) has the solution 

p(x) = ^e-( >u <-*\ Z= ( e~ 0u dx (10.7) 

Jo 

and the corresponding stationary current is j p = 0; this is a detailed balanced 
dynamics with p the equilibrium density. 

When adding a nongradient driving force, J Q fdx ^ 0, (|10.6p can still be solved; 
the stationary density obtains the form 

I f 1 e f>W(y,x) pi pi /3W(y,x) 

~/ Fv \ dy, Z= / ———dydx (10.8) 



ZJo D{y) y ' J a J D(y) 

where 

{\ x f dz for y < x 

J y f dz + J Q fdz for y > x 

is the work performed by the applied forces along the positively oriented path 
y — > x. In this model the stationary current can be computed by dividing the 
stationary equation (|10.6[) by px and by integration over the circle: 

W 

Jp = — j — (10.10) 

Jo Ox) 1 dx 

where W = fdx is the work carried over a completed cycle. The non-zero 
value of this stationary current indicates that time-reversibility is broken. In the 
simplest noncquilibrium setting when [7 = and /, x > are some constants, the 
steady state has the uniform density p(x) = 1 and the current is j p = xf ■ 



10.2 Entropy and traffic 

Before going on to defining the observables for which we want to describe large 
deviation principles, let us first discuss the basic ingredients that play a role in 
the rate functions: entropy and traffic. 
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To properly define them, we need to define a reference process. We take this 
process to be (see 14. 7|) : 

dx t = V ■ D(x t )dt + ^2D(x t )dB t 

i.e. the overdamped diffusion (|ll).2p with the force / — VZ7 put to zero. The 
action that describes the probabilities of trajectories u) = (x t )o<t<T of the original 
process with respect to the reference process is given by (|4. 15|) : 

e -A H = ggo (w) (lon) 
al Mo 



exp 



[ll dt[2x t + 2V • D — X (f - VC/)] • D~\{f - VU) | 



10.2.1 Entropy 

The entropy flux into the environment is equal to the time-antisymmetric part of 
the action (by local detailed balance, see (|4.22[) ): 



S(w) = A{Bw) - A(w) = p J dx t o (f(x t ) - VU(x t )) 

= p f dx t o f(x t ) - p[U{x T ) - U{x )} (10.12) 



which is equal to the work done by the nonconservative force (see (|4.21|) ) minus 
the change of energy of the system. It is useful for later calculations to compute 
the average of the entropy flux, started from some initial distribution /io- For that 
we need to compute the average of the Stratonovitch integral. We rewrite the 
Stratonovitch integral to an Ito integral using (|4.19j) : 

f dx t of(x t )= f dx t f(x t )+ f dt[D{x t )V]- f(x t ) 
Jo Jo Jo 

Using the Langevin equation, the average of the Ito integral ( dxtf(xt)\ gives: 

> 'Mo 

T 

(f(xt) ■ (x(xt)lf(x t ) - VU(x t )]dt + V • D(x t )dt + y^2D{^t)dB t ) ) 

Q \ \ / / jLl 

T 

dt(f(x t ) (x(xt)lf(xt) - VU{xt)\ + V • D(x t )) ) 



ENTROPY AND TRAFFIC 



127 



In the last step we used that the average over the term with dB t drops, because 
dB t = Bt+dt — B t is independent of x t and has average zero. Then, by definition 
of the time-evolved distribution [i t we get 



dx t o f(x t ) 



J dt J dxfit (/ ■ X [f - VC/] + V ■ [Df] 



which can be rewritten succinctly, using partial integration, as 



dx t o f(x t ) 



dxdtf ■ j Mi 



(10.13) 



where j Mt is the probability current as in (|10.3p . The average entropy flux, starting 
from a density (1q is thus equal to 



<SM>„ n = / dt 



dxf ■ j Mt - -r I dxU(x)/j,t{x) 



(10.14) 



In Section [2. 81 we also discussed the average entropy There we defined the entropy 
of the system through the Shannon (or Gibbs) entropy: 



dxfi(x) log n{x) 



The total average change of entropy (entropy production) in the world, given the 
initial distribution /io is then, see ()2.18|) : 



(S M ») W) = / dta(vt) (10.15) 
<J o 

where the instantaneous entropy production rate cr(/i) is easily computed to be 



cG«) = in' (M-D) 1 3» dx 



(10.16) 



It is very important to note that, although computed through the action relative 
to a reference process, the average entropies computed here do not depend on that 
reference process (as long as the reference process is an equilibrium process). 
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10.2.2 Traffic 

The traffic is defined as the time-symmetric part of the action (|1U.1 ip : 

T(w) = A(6u) + A(u>) (10.17) 

= C f T dt ^ ~ VC/ ) • D if -VU) + /3 f dtV ■ [D(f - VU)] 
1 Jo Jo 

The average value of the traffic, given an initial distribution /iq is then equal to 

(T(w)) M0 = f dtr^t) 
Jo 

with the instantaneous 'traffic rate' given by 

r ( M ) = L. J dxfiif - VU) ■ D(f - VU) + P J dx/xV • [D(f - VU)] (10.18) 

The traffic, in contrast to entropy, is very much dependent on the choice of the 
reference process. It is interesting to note that for over damped diffusions, the 
entropy rate and traffic rate are closely related. To see this, note that the entropy 
production rate for the reference process, can be found by simply putting / — V/7 
to zero in the explicit formula 1)10.160 : 

o°M=f*c<™- D <™ 



One can then check directly that 
ajfx) -a°{fj,) 

nW = g (10.19) 

In following sections we often compare the traffic rates of different dynamics 
(determined by different forces). This equality then tells us that these traffic 
differences can be replaced by entropy differences. Hence, relation 1)10.190 brings 
about a simplification in the structure of fluctuations that is characteristic and 
restricted to diffusions. See for example the next chapter, where there is no relation 
like (|1O.190 . We believe that this also indicates that a more general noncquilibrium 
theory should reach beyond the Langcvin or diffusion approximation. 
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10.3 Observables 



Here we define the observables for which we want to compute the rate functions. 

A basic and time-symmetric dynamical observable is the empirical distribution of 
the occupation times. It is defined as the fraction of time spent in a subset A of 
the state space Q over a fixed time interval T: 



where xa(%) is the indicator function giving a value 1 if x G A and zero otherwise. 
Note that this resembles the definition of the empirical occupation vector (|9.ip 
in the example explained in the last chapter. The difference is that the particle 
number ./V is replaced by the duration time T and the state space is continuous. 

Having this in mind we formally write the empirical occupation density p: 



This is a path-dependent observable as it varies over the paths u = (xt)o<t<T- 
Note that it is normalized (i.e. integration over state space gives one). It is also a 
useful observable, because we can write time-integrated state-functions g in terms 
of this empirical density: 



Because we always assume that there exists a unique stationary distribution, we 
have ergodicity: 



almost surely for T — > oo for any state function g (this means that the probability 
to have a path uj for which this convergence holds is equal to one). We conclude 
that for large times p u — > p almost surely. 

The time-antisymmetric observable of special relevance is the empirical current. 
We define the total time-averaged current in a set A C f2 as 





(10.20) 
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where the circle denotes a Stratonovitch stochastic integral, and \a is again an 
indicator function. This observable is the sum of all displacements the particle 
makes when in a subset A of f2 and thus represents the time-integrated particle 
current (per unit of time). As usual, there can be an ambiguity in choosing the 
type of stochastic integral. The Stratonovitch integral is there to assure that the 
observable is time-antisymmetric. The current density is then formally defined as 



It depends again on the (random) path u and it measures the time-averaged current 
while in x. As for the empirical occupations, this observable is useful because 
quantities like work and the entropy flux can be written in terms of it: for an 
arbitrary state function g we get 



In the same way as for occupations and with ()10. 13|) we see that J u — > j p for 
T — > oo almost surely. 

10.4 Fluctuations of occupations and currents 

We already argued that the empirical densities of occupations and currents will, in 
the large time limit, converge almost surely to the stationary densities p and j p . In 
this section we compute the rate functions (fluctuation functionals) for deviations 
from these typical values. This amounts to computing the probabilities that the 
empirical observables p u , J w are close to some given but arbitrary density fj, and 
current j. 

10.4.1 Definitions and restrictions 

To be able to compute the probabilities of the empirical observables, we have to 
define the distance d between p u and [i and between J w and j. With such a 
definition, probabilities like P(d{p u , fi) < e) for some small number e > should 
be computed. Actually proving that the large deviation principle holds, i.e. 




(10.21) 




lim - log P(d(pu,fJ.) < e) = + O(e) 



(10.22) 
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and under what restrictions, is not trivial. In this text we do not concern 
ourselves with this. For mathematical details, see [231 [27] HH1 HE ISO] ■ Instead, we 
assume that the large deviation principle holds, and concentrate on computing the 
rate functions using heuristic arguments. We begin by introducing the following 
simplified notation for ()10.22p : 

- lim i log P{ Pu = fj.) = or P( Puj = M ) x e- TI ^ 

T— yoo 1 

Similarly as for the occupations the probability of deviations for the currents are 
defined. However, the natural starting point is the joint probability for currents 
and occupations: 

P(p u =l*,J u =j)~e- TI ^ 

as both and can be obtained in principle from the joint fluctuations by 
the contraction principle. 

A first observation that we can make, is that I(p,j p ) = 0, i.e. the stationary 
density and current have a probability that does not decay to zero. Moreover, the 
stationary regime is characterized by 

I (p,j P ) = inf. 

A second observation is that /(^, j) = oo = I(J) whenever j is not stationary, i.e. 
for V • j =/= 0. Indeed, for any smooth bounded function Y one has 

jYV-J u da: = ~J VY(x t )odx t = -^[Y(x T )-Y(x )}^0 

for T — > oo. Hence, in a distributional sense, V ■ J w — > for T — > oo along any 
particle trajectory, which proves the above statement. That is why from now on 
we always assume that V • j = 0, unless otherwise specified. 



10.4.2 Computation of the fluctuation functional 

The probability of the fluctuations can be written as 



P(Pu = P, Ju = j) = J dP{u)x[Puj = Mi Ju> = j] 
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where x is again an indicator function, giving one when its argument is satisfied 
and zero otherwise. We have omitted initial conditions in the notation, as they 
are not important in the long-time limit. (To be more rigorous, one has to define 
a neighbourhood of [i and j using the distance defined on the function space. 
The indicator function then gives one when the empirical densities are in that 
neighbourhood.) To compute the rate function we exploit the knowledge we have 
of Radon-Nikodym derivatives. For this we define a new dynamics, by changing 
the force / in (|1U.2|) to a new force g. We take this g such that the density \x and 
the current j become typical, i.c: 

j = xlg - vu]fi - DV/i 

This explicitly defines g. With this new dynamics we write: 

P(Pu = fJ<,Ju = j) = J dP g {u)^f{u) X \p u = fi, Ju = j] (10.23) 

where we now have added subscripts / and g to denote respectively the old and 
the new dynamics. The Radon-Nikodym derivative can be written in terms of the 
entropy production and traffic: 

dPt 

log -J- M = s f m - s 9 (uj) + r» - r f (w) 

which arc defined in ([10.12|) and (|10.17p . Again the quantities with a subscript 
g are defined in the dynamics with / replaced by g. The excess entropy can be 
rewritten in terms of the empirical current: 



S f (u) - S g {co) =/3jdx t o [f[x t ) - g(x t )] =!3TJ dxJ u {x)[f{x) - g{x)] 

The excess traffic can be rewritten in terms of the empirical occupations: 
7» - 7>(w) = PT[tM ~ Tf{p u )] 

where r g is given by (|10.18|) . but with / replaced by g. We see that the entropy 
production in both dynamics only depends on the path u through the empirical 
current, and the traffic in both dynamics only depends on the empirical occupation 
density. When we substitute this in (|ll).23p we see that the indicator function 
allows us to replace the empirical observables by the density fi and the current j. 
We thus get 

P{Pu =H,J U = j) = e~ TI ^ f dP g {u>)x\p* = (i,Ju> = j] (10.24) 
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where the functional /(/i, j) is given by 

Ifaj) = \r f {n) - \tM + dxj -(g-f) (10.25) 

This functional is exactly the rate function, because the last factor in (|10.24|) goes 
to one in the limit of large times, which is a consequence of the fact that [i and j 
are the typical (stationary) values in the dynamics determined by g. 

Let us examine this joint rate function J(/x,j). It is the sum of an excess 
instantaneous traffic rate given the density fi alone, and an excess work (or, 
equivalcntly, entropy flux) given stationary current j alone. The excess is an 
excess of the original dynamics with respect to the dynamics in which [i and j are 
typical. If we use ()10. 19|) . we can rewrite the functional in terms of the entropy 
production rate: 

J(/i, j) = \<T f (fi) - \o g {n) + dxj ■ (g - f) (10.26) 

Furthermore, using the explicit expressions of the traffic/entropy and the force g 
we can rewrite the fluctuation functional in the following explicitly positive form: 

Hfrj) = J°70) - J^ffO) + - J dxj ■ - hi) 

= \JU~ J'm) ■ W\3 - Jm) dx (10.27) 

(On the assumption V • j — 0; remember that I(fi,j) = oo otherwise.) This 
last formula resembles the Gaussian- like expressions for the current distribution, 
typical for hydrodynamic fluctuations of the diffusion- type. Such expressions are 
omnipresent in the works of e.g. [7J Although the quadratic integrand in 
(|10.27|) resembles the (generalized Onsager-Machlup) Lagrangian for macroscopic 
fluctuations in the hydrodynamic limit, we have no spatial/ temporal rescaling here. 
We have started from a mesoscopic system as described by a diffusion equation and 
the only large parameter is the time span T. The difference between our work and 
other work on dynamical large deviations becomes visible from the formula (|10.25D 
and (|10.27|) . Our approach has nothing to do with hydrodynamic rescaling or with 
macrostatistics. It concerns the thermodynamic interpretation of the fluctuation 
functional /(/i,j) for our mesoscopic system: how it is shaped from quantities 
like traffic, work and entropy production, and providing full account of the steady 
dynamical fluctuations in both the time-symmetric and the time-antisymmetric 
sectors. 
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10.5 A notion of thermodynamic potentials 

In equilibrium systems it is useful to consider thermodynamic potentials (like free 
energy), as they have a clear physical meaning and characterize equilibrium via 
variational principles. Moreover, to go from one potential to another, Legcndrc 
transforms are used. Fluctuation functionals also bring with them variational 
principles. First of all, and mainly, minimizing them characterizes stationarity. 
In (|10.25p we gave an explicit expression for the rate function of occupations and 
currents. Let us analyze this from the viewpoint of thermodynamic potentials 
and Legendre transforms. First of all, the first two terms in ()10.25p constitute an 
excess traffic, or cquivalcntly excess entropy production (|10.19[) . Let us consider 
the traffic in a dynamics governed by an arbitrary force h: 

Th (fi) = L. J dxfi(h - VU) ■ D(h - VU) + (3 J dxfiV ■ [D(h - VC/)] 

This can be seen as a potential for the currents in the sense that the functional 
derivative of it with respect to the force, gives 

PfJx) (10.28) 



Sh(x) 

where j£ is the probability current in a dynamics with a force h, see (|10.3|) . It is 
then natural to examine the Legendre transform of r^: 

G{n,j) = sup /3 J h ■ jdx - T h (fi) 

We find the suprcmum by using Eulcr-Lagrange equations: 

i = # 

so that h = g is exactly the force needed to make the current j typical, together 
with fx: 



G(jJ>J) = P j 9- jdx- T g {n) 
On the other hand, if we take the functional derivative of G, we get: 



(10.29) 
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where g is again the force that makes /i and j typical. So G is a potential for the 
forces, just like r was a potential for the currents, and by Legendre transforms we 
can switch between the two. 

We can easily rewrite J(/z, j) in terms of G and r: 



*(M.i) = 2 



(10.30) 



Note that to(/z) = 0, so that iG(/z, j) is exactly equal to the fluctuation functional 
in the case that / = 0. In other words: 



dxf ■ j 



This is nice, because the left-hand side is a fluctuation functional for a 
noncquilibrium dynamics, while on the right-hand side, the first term is a 
fluctuation functional in an equilibrium dynamics. The rest is thus 'the correction 
to equilibrium.' 

We can still rewrite (|10.30|) in the form 



47 / ( M , j) = sup{2/3 / dxf ■ j - a f(ii)} - 2/3 / dxf ■ j + a f (p) 



fully in terms of cntropic quantities, due to (|10.19p . A similar structure, cf. 
(110. 28|) and (|10.29p , has been established already before in the framework of jump 
processes, see [BS] and also the next chapter. 



10.6 Contractions 

Now that we have a fluctuation functional for both symmetric and antisymmetric 
variables, we can compute the statistics of empirical averages of arbitrary physical 
quantities. In particular, we can try to find the fluctuation functionals for density 
J(/x) and for current separately. To start we look at the fluctuations of the 
occupation density. 
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10.6.1 Occupation statistics 

As = infj we have to compute the minimizing current j for any given 

density fi. Since the minimization is constrained via the stationary condition 
V ■ j = 0, we get the Euler-Lagrange equation from (|f 0.27|) 

i=XM[/-V-(C/ + y)] -DWfi (10.32) 

where V is a Lagrange multiplier (function of x). Not surprisingly, we sec that 
the minimizcr is the stationary current for a modified dynamics that makes y, 
stationary. This modified dynamics is achieved here by adding to the imposed 
potential U an extra potential V. We therefore call the minimizing current in 
(|10.32[) and the fluctuation functional becomes: 

m = \j Ctf - Jm) • ~ J,) dx (10.33) 

For some explicit examples of solutions to ()10.32|) . see further down, in equations 

snrsn and mm . 

The fluctuation functional obtains other equivalent forms by substituting 

.9 = / - W into l|ffi2% 

^ = rQQ - tv(m) = <M - ovM (1Q 34) 

where the second equality follows again from ()10. 19[) . In this way we have 
recognized the excess traffic (or here also: the excess entropy production) as 
governing the large time statistics of the occupation times. Excess is here excess 
of the original with respect to the modified process in which /i is typical. The 
fluctuation functional I(y) thus exactly equals one quarter of a difference in 
entropy production rates when having density (i, these rates being computed 
respectively for the original dynamics and for a modified dynamics that makes 
H stationary. 

In formulas (|10.33p and ()10.34|) the potential V has to be determined from /j, by 
solving the inverse stationary problem (|10.32p . We discuss two examples in which 
this can be done explicitly: the equilibrium case and the example of diffusion on 
the circle. 

Equilibrium dynamics Let us see what can be said in general for equilibrium 
diffusions. If / = then equation ()10.32p has the solution V = — /3~ 1 log/i, and 
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the corresponding current jY and the entropy production ay (/i) are both zero. As 
a result, 

J(A0 = ^ (10.35) 

This exact relation between the equilibrium dynamical fluctuations and the entropy 
production is solely true for diffusion processes. In contrast, for jump processes (see 
the next chapter) cr(/i) gives only the leading term in an expansion of /(/i) around 
the equilibrium density p cx e~@ u , and the relation (|10.35p obtains corrections 
when beyond small fluctuations; sec (68 for details. 



Diffusion on the circle For the one-dimensional example of Section 110.1.21 the 

inverse stationary problem (|10.32p allows for an explicit solution. The current 
is immediately read off the formula (|10.10p , 



jl= 1 — , W= I fdx (10.36) 

J (pD)~ 1 dx Jo 

and the potential V obtains the form 

V(x) = -U(x) - i log^(x) + J\f - jj^) dy (10.37) 

which is a nonlocal functional of the given density \i. The fluctuation functional 
is explicitly given as 

w 2 

4J(/z) - a{n) - -t— — — (10-38) 
J {fxD) 1 dx 

for n 7^ 0. 

Observe that if fx = on some open set A then the rate function equals = 
a(fj,)/4. (That follows also from the equilibrium form (|10.35|) below as the circle 
gets effectively cut and the dynamics mimics a detailed balance one.) The infimum 
of I(fJ-) over all densities fi that vanish on A then gives the escape rate from the 
complement A c — [0, 1) \ A. 

As a simple example, assume that U — and let / and D be some constants. In 
this case the entropy production (|10.16p reads 

<j(p)=p 2 Df + D [ ^—dx (10.39) 
Jo M 
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To compute the escape rate from A c (or, entrance rate to A) we must take the 
infimum of (|10.39p over all fi that vanish on A. Setting A = (0, S) for some < 
8 < 1, that infimum is reached for the density fi*(x) = j^rg sin 2 ( *^Zp )■> x G [$i 1]; 
and the escape rate is 

, / x / *x k 2 D 8 2 Df 2 
inf Ifa ) = I(H* ) = —— + ^-i- 10.40 
mU=o (1 — oy 4 

Even in equilibrium (/ = 0) the result is meaningful as it relates the diffusion 
constant to an escape rate. In the context of dynamical systems, the analysis of 
the escape rates and of their link to linear transport coefficients was initiated by 
Dorfman and Gaspard, see [3TJ [32] and references therein. 



10.6.2 Current statistics 

The contraction to the current j is also possible. However, up to special examples, 
there is no explicit solution to the associated variational problem and for general 
models one has to resort to a perturbative or numerical analysis. In fact, often 
the calculation starting from the generating function of the current appears more 
practical than to do the contraction starting from I(fi,j), see e.g. [25]. Through 
the Gartner-Ellis theorem the generating function is related to the fluctuation 
functional via a Legendre transform. 

However, we can check in general a fluctuation theorem for currents: notice that 

-j) - = \ J dxj^D)- 1 ]^ = - J dxj ■ f 

This remains true even when the contraction to currents alone is made. We thus 
have that 

" ex P i / dx i ■ f 



P{J U = -j) — \ 2 

We see that currents that have on average the same direction as the forcing, are 
exponentially more probable than their reversed currents. 



For explicit computations of the current rate function we restrict us here to giving 
the result for a constant drift on the circle. 
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Constantly driven diffusion on the circle We take U = and /, x constants. In 
this case, from (|10.27p the joined fluctuation functional reads: 

= 4^ / ~(i - P D f» - D ^') 2 dx ( 10 - 41 ) 

and for all j, the infimum over p is reached at the uniform distribution, so that 



m = ™^ (10.42) 
and hence we see that here the current fluctuations are Gaussian. 



10.7 Small fluctuations and entropy principles 

Considering small fluctuations (for which a quadratic approximation is valid) can 
simplify the fluctuation functionals mathematically. It is also experimentally 
a more accessible regime, as the fluctuations are more probable than large 
fluctuations. The fluctuation functionals can be expanded in both the occupation 
densities and currents around their typical values and the strictly positive 
quadratic form obtained in the leading order describes normal (Gaussian) 
fluctuations. From a physical point of view, the structure of these normal 
fluctuations have been first analyzed by Onsager and Machlup, [57] . for the case 
of relaxation to equilibrium. Here we show a natural extension of the original 
Onsager-Machlup formalism to noncquilibrium systems by starting from the above 
fluctuation theory. 

We look here at the Gaussian approximation in a dynamics far from equilibrium. 
Later we will also make the driving / small, to be close to equilibrium. 
As is clear from (| 1 (J . 2 7 p , current and occupations are coupled. It is because of this 
coupling that contractions of I(p, j) to I(p) and to become rather complicated. 
Even for small fluctuations this coupling remains: take p = p(l + ep\) and j = 
j p + eji, with e a small parameter. Because j — j p is then O(e), the fluctuation 
functional is 0(e 2 ): 

e 2 f 

I(f*J) = j J dx[j!-(pD) + i4j p -(pD) l j p 

+V M i • pDVm - 2 M iii • { P D)- l j p } + o(e 2 ) (10.43) 

The last term in the integrand gives the coupling between occupation and current 
fluctuations. It is proportional to the stationary current, which is non-zero away 
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from equilibrium. It is only when we take a dynamics close to equilibrium, i.e. 
/ = e/i, that the fluctuations decouple. In this approximation we have that 
j p = O(e), and thus, near equilibrium, 

= jf dx[h ■ { P D)- l 3l + Vmi • pDVm] + o(e 2 ) (10.44) 

with, to leading order, a complete decoupling between the time-symmetric 
(occupations) and the time-antisymmetric (current) sectors. 



Occupations When close to equilibrium, the computation of I(p) by contraction 
is easy: we see from (|10.44|) that the second term on its right-hand side is just 
J(M). 

In the same approximation of small fluctuations and close-to-equilibrium, the 
entropy production becomes: 

<t(m) = / dx [j p ■ {pD)-% + e 2 V/ii • pDV^} + o(e 2 ) (10.45) 

and thus we get 

/(,) = ^ T ^ + ( £ 2 ) (10.46) 

This reveals to be a special case of a general result, [BS], according to which the 
entropy production governs the occupational statistics in the linear irreversible 
regime. It provides a fluctuation-based explanation for the minimum entropy 
production principle introduced by Prigoginc to characterize stationarity via an 
(approximate) variational principle, |89j : the stationary state has a minimal 
entropy production, which is not necessarily zero because the system is out of 
equilibrium. 



Currents For currents we have an analogue of the minimum entropy production 
principle. The starting point is again (|10.44|) from which we extract the current 
fluctuations: 



- I l(j-j p )-{pD)-\j-j p )dx + o{e 2 ) 



-[V(j p )+V(j)-2S(j)]+o(e 2 ) 



(10.47) 
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with T>(j) — J j ■ (pD)^ 1 j dx sometimes called the Onsager dissipation function, 
and S{j) — ft J f -jdx is the entropy flux given a current j. In particular, this leads 
to a variational characterization of the steady current j p which can be written 
as the following maximum entropy production principle: the j p maximizes the 
entropy flux S(j) under the two stationary constraints 

(l)V-j = 0, (2)V(j)=S(j) (10.48) 

The second condition is indeed satisfied at j = j p (note also that p can with no 
harm in this order be replaced by the equilibrium density p/=o = e~° u /Z.) Such a 
variational principle, known as a maximum entropy production principle, is often 
used in applications and apparently even beyond the linear irreversible regime. As 
is however clear from (|10.47|) from our dynamical fluctuation theory, the validity of 
the maximum entropy principle is restricted to close-to-cquilibrium. Beyond that 
regime, we must refer to contractions from ()10.27|) . (|10.30p or even from (|10.43p 
for generally valid expressions with a general thermodynamic meaning. 



Chapter 11 

Markov jump processes 



Here, the large deviation theory is applied to Markov jump processes, again with 
empirical occupation and current densities as observables. The lines of reasoning 
are the same as in the last chapter, and therefore the discussion in this chapter is 
less detailed. For more details, we refer to i7U\j . 



In this chapter we consider time-homogeneous Markov jump processes on a finite 
state space f2, determined by the transition rates k(x,y). We imagine the system 
to be in contact with a single heat bath. The local detailed balance assumption 
then restricts these rates (|3.10[) : 



where U(x) is the energy of the system in state x and W(x, y) = — W(y, x) is the 
work done on the system by a nonconservative force during the transition x — > y. 

Again the main ingredients for the dynamical fluctuation theory are the entropy 
and traffic, which are defined through the path-probability density, relative with 
respect to a reference process. As the reference process we take an equilibrium 
process with rates k (x,y) defined through 



11.1 Entropy and traffic 



k(y,x) 



e -P[U{y)-U{x)-W{x,y)} 



k(x,y) = k {x,y)e 



(11.1) 
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One can check that the rates ko(x,y) are detailed balanced with equilibrium 
distribution p(x) cx exp[—/3U(x)]. The action of the original process with respect 
to this reference process is given by (|3.9|) : 



-A{u>) 



i dV ( \ 



dt[Xo(x t ) - \(x t )] + ^ log 



t<T 



Hx t -,x t ) 
k (x t -,x t ) 



where the sum is over all jump times of the path, x t - is the state just before the 
jump and X(x) = ^ y k(x,y). Using the relation between the original and the 
reference process, this becomes 

-A(w)= / dtj^k (x u y)[l-e% w ^} + ^ ^Vfo-.a*) (11.2) 

JO y 



11.1.1 Entropy 

The excess entropy flux (original system with respect to the reference) into the 
environment during a trajectory to = (xt)o<t<T is given by 



t<T 



For later purposes we would like to compute the average of the entropy production. 
We prove that for any function of two states g{x, y) and for any initial condition 
(jl: 

(^2g(xt-,xt)j = I dt^2m{x)k{x,y)g{x,y) (11.3) 
\t<T / J ° x,y 



We do this by defining yet another set of rates k*(x, y) = k(x,y) exp[h ■ g(x,y)], 

dV* 
dV 



with h € K and using the identity 1 = ( ^-(w) 



exp ■ 



dt^Tk(x t ,y)[l 
y 



hg(x t ,y] 



] + h^2g(x t -,x t ) 



t<T 



/'(> 
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Taking the derivative with respect to h in h = of this equation gives us then 

i-T 







/ dt^2k(x t ,y)g(x t ,y) + ^2g(x t -,x t )) 
y *< T / w 



which proves (|11.3|) . The average of the excess entropy flux is thus 

i-T i pT 



))uo= f dt^2^ t (x)k(x,y)W(x,y) = \ [ dt^2j flt (x,y)W(x,y) 

•'" x.y " x,y 

with jfj,(x, y) = fji(x)k(x, y)—fj,t(y)k(y, x) the probability current. This entropy flux 
is an excess with respect to the reference equilibrium process. For the equilibrium 
process we have, because of detailed balance: 

S (u) = log = -P[U{xt) - U(x )} 

The average of this can be written as 

X X y 

In Scction ETSl wc defined the entropy of the system through the Shannon (or Gibbs) 
entropy: 

s(fx) — — J dxfj,{x) log fx(x) 

The total average change of entropy (entropy production) in the world, given the 
initial distribution /zo is then the sum of the entropy flux of the reference process 
plus the excess entropy flux plus the entropy change of the system, see also (|2.18jl : 

(^ H) Un = S (/i T )-s(/ io ) + ([5o(a;)+5 e:c ( W )]) /lo = dta(m) (11.4) 



where the instantaneous entropy production rate cr(/i) is straightforwardly 
computed to be 

"•(AO = l^j»(x,y)[ l °sKx) -\o gl x{y)+pU{x) ~(3U(y) + W{x,y)] 
= ^ { x)kix,y)lo Em ^ 
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11.1.2 Traffic 

As for the entropy flux, we get the excess traffic from the action: 

T ex {uj) = A(9cj) + A{u) = 2 f dt[X(x t ) - Ao(ar t )] (11.5) 

Jo 

The average of this, starting from an initial distribution /j,q, is 

{T ex ) m = 2 / dt[r((x t ) - T (fi t )] 
Jo 

with 

t (m) = V{x)K x ) = fi(x)k(x, y) (11.6) 

x x.y 

which is equal to the average number of jumps per unit of time for a system in a 
distribution /i. We call it the activity of the system. Note that this activity can 
be seen as the instantaneous traffic rate (only with a different reference than in 

(HD). 



11.2 Observables 

Like in the last chapter our first observable is the empirical time-integrated 
occupation density: 

1 f T 

Pu>{x) = — J dtS XuX 

where 6 Xt ,x is the Kronccker delta, giving one when x t — x and zero otherwise. 
Furthermore we define the empirical distribution that counts jumps: 

Qu{x, y) = ^ Y Sx t- ,* S *t,y 

t<T 

Again, as for the case of diffusions, ergodicity leads us to the conclusion that in 
the long time limit p u — > p and Q u {x,y) — > p{x)k(x,y) almost surely. From 
we can define the empirical current: 

Ju(z, y) = Q u {x, y) - Q u {y, x) 

which converges in the long time limit to j p almost surely 
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11.3 Joint fluctuations of occupations and currents 



The easiest starting point for deriving fluctuation functionals is the joint 
fluctuations of the occupations and jumps: 



P{Pu> = H, Qlj = q) x e 



-TI(»,q) 



Again, one has to take care in correctly defining such a large deviation principle, 
and prove when it is valid. We will not do this here. Instead we concern ourselves 
with computing the rate functions. We refer to [H] for mathematical details. 

The heuristic argument to compute the rate function /(/i, q) is essentially the same 
as the one in the last chapter. We first define a new dynamics in which q and [i 
are typical, i.e. we take a process with transition rates q(x,y)/ fx(x). Then, we 
compute the action for a path u for the original process with respect to the new 
one: 

q{x t ,y), , v^. q(x t -,x t ) 



H(x t -)k{x t -,x t ) 

Using the definitions of our empirical observables, we rewrite this to 

AM =Tj2{p^)[k{x,y) - + Q u (x,y)log q( £ v) \ 

jr£\ MO) n(x)k[x,y) J 

We then see that the probability of the fluctuation becomes 
P{Pu =(X,Q u = q)= I dV q (oj)e- A ^ X [Pu =H,Q U = q] 



And, similarly as for diffusions we can replace the empirical observables in the 
action by jj, and q, so that in the end the rate function becomes 

7 0"> «) = E y) ~ q(x, y) + q(x, y) log } (H-7) 

However, we would like to find the rate function for the joint fluctuations of 
occupations fi and currents j. By the contraction principle, we see that this rate 
function is the infimum of I([i,q) over all q for which q(x,y) — q(y,x) = j(x,y). 
Note that for any path u> and any (bounded) state function Y, we have that 



Y(x )-Y(xt) 
T 
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which goes to zero for the long time limit. Therefore we only consider fluctuations 
for which j(x,y) = 0. The q* that makes the infimum can be explicitly 
computed. For that we compute the Euler-Lagrange equations: 

q{x,y) = ^i{x)k{x,y)e m ^ 1 (11.8) 
where F(x,y) = —F(y,x) is a Lagrange multiplier, which makes sure that 

j(x, y) = q(x, y) - q(y, x) = (i(x)k{x, y)e 2 - ^(y)k(y, x)e 3 (11.9) 

This equation can be explicitly solved, giving the minimizcr q*: 

j (x yj 1 

<l*{x,y) = — j h 7j V b'0,2/)] 2 +^{x)n{y)k{x,y)k{y,x) 

This can then be substituted into I(fi,q = q*) to get I(fi,j). Going beyond the 
mathematics, we compare ([11.80 to the local detailed balance assumption which 
says that 

y( x >y) = K x ) c -0\u(v)-U(x)-W(x,y)-F(x,v)] 
q(y,x) ti(y) 

This gives an interpretation to the Lagrange multiplier F: it is a modification of 
the work W(x,y) originating from a nonconservativc forcing. We therefore call F 
a force. Moreover, the equation (|11.9|) tells us that this force makes the fluctuation 
/i, j typical. This is the same kind of modification as we had in the last chapter. 
There, to find the joint rate function of occupations and currents we also had to 
introduce a new force which made the fluctuation typical. It is illustrative to write 
in terms of this F by substituting (|11.8p into J(/n, q): 

= ^2iM{x)k(x,y)[l-e^ p ^] + ^j{x,y)F(x,y) 

x,y x,y 
x,y 

In the last line we have written the excess activity of the system, excess meaning 
the modified dynamics with respect to the original one. Similarly, the last term in 
the rate function is an excess entropy flux, given a current j. This last expression 
coincides physically with ([10.250 . and is as such the most general and physical 
form of writing the rate function. 
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We can also use the reference equilibrium process to rewrite the fluctuation 
functional into a form even more clearly similar to (|10.25l) : 



+ j^23^,y)[G{x >y )-W(x,y)] 
= Tw(fjt) - r G (/i) + -^2j(x,y)[G(x,y) - W(x,y)} 



where G — W + F is the nonconscrvative forcing in the modified dynamics, and we 
have denote subscripts W and G to denote the traffic in the original and modified 
dynamics. 



11.4 A notion of thermodynamic potentials 

Like in Section 110.51 we can use the traffic of a process to construct a potential. 
The activity, as defined in (|11.6|) . has the following property: define a dynamics 
with rates kh{x,y) — ko(x, y) exp{^h(x, y)}, where h(x,y) = —h(y,x). then the 
function H(fi, h) = Th{n) — To (fj,) satisfies 

^(x y) = f lv(x)kh(x,y) - n(y)k h (y,x)] = ^(x,y) 

meaning that the activity (traffic) is a potential for the currents, like the traffic 
was in the overdamped diffusion case. Analogous to that case we can define a 
Legendre transform: 



G(fj,,j) = sup[-^2 j(x,y)h(x,y) -H([i,h)} 



This is a potential for forces: the minimizer h is found to be the force G = W + F 
that makes j = jf* typical, and 

SG p 

= ^G{x,y) 



Sj(x,y) 2 
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The fluctuation functional 7(/i,j) can then be written as 

Exactly like for the diffusion case (|ll).3Up . Again, G(fi,j) is equal to Iw=o(l^,j), 
i.e. the equilibrium fluctuation functional. 



11.5 Contraction to occupations 

To find the fluctuation functional for occupations alone, we make the contraction 
J(/i) = inf j) or = inf q) 

3 1 

The infimum has to be taken with the restriction that ^2 y [q(x, y) — q(y,x)] = 
^2 y j{x,y) = 0. If we do the contraction using I[fj,,q), we get Euler-Lagrangc 
equations of the following form: 

q(x,y) = n(x)k(x iy )e? [v(y) ~ v{x)] 
y 

where V is a Lagrange multiplier making sure that ^2 y [q{x,y) — q{y,x)] = 
Y^ y j( x ^y) = 0- Although we can not solve this in general, we see that V is a 
potential to be added to the dynamics that makes (i typical. This is analogous to 
the overdamped diffusion case. Substituting this form of q in q) we can write 
/(/it) in terms of this potential: 

= E {MaOfcfo V) ~ y)eM v ^~ v ^ } = r( M ) - r v ( M ) (11.11) 

x.y 

Indeed, the fluctuation functional for occupations alone is defined in terms of an 
excess traffic, just as for the case of overdamped diffusions. In contrast to that 
case, however, this excess is not equal to an excess of entropy production. It 
seems therefore, that the excess traffic is a more general quantity than entropy to 
characterize fluctuation functionals. 
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11.6 Small fluctuations 

Consider the regime of small fluctuations, i.e. where the quadratic approximation 
of the fluctuation functionals is valid. We write fi(x) = p(x)[l + e/zi(x)] and 
j(x,y) = j P {x,y) + eji(x,y) with e S R a small number. Note that j\ has to 
satisfy the constraint j i(x,y) = 0. To write down up to second order 

in e, we expand first (|11.9|) . giving F(x, y) = eF±(x, y) + o(e), with 



where we have defined the symmetric counterpart of the current: 

t~h(x, y) = p(x)k(x, y) + p(y)k(y, x) 

which is the expected number of jumps between states x and y per unit of time, 
when the system is in a distribution /i. This quantity is closely related to the 
activity defined in (|11.6p . because 



The factor of two in this equality stems from the fact that every bond x, y is 
counted twice in the sum. We therefore also call r p (x,y) traffic (or activity). The 
fluctuation functional /(/i, j) in terms of the added force F becomes 



Fi(x,y) 



1 



h{x,y) - ■^r p {x,y)\nx{x) - (^(y)] 



T p {x,y) 



■^jp{x,y)\pi{x) + in(y)) 



^2r,,(x,y) = 2t(» 




YjTpix^F^y)] 2 +o(e 2 ) 



8 



and substituting the explicit form of F, we get schematically: 




l 



TpV /ii - j p \7+^ 1 ] 2 (x,y) + o(e 2 ) 



(11.12) 



with the notation V ± ^i(a;, y) — 5 [fii(x) ±/Zi(y)]. In this quadratic approximation, 
we see that the stationary traffic t p plays the role of a variance. 
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We can write ()11.12p in the form 

= ^E[|- + y( v "^ 2 -^-v+Mi + ^(vWl(^y) 

x,y P P P 

+o(e 2 ) (11.13) 

which demonstrates that there is an occupation-current coupling, which is 
proportional to the stationary current. It is this coupling that makes contractions 
to occupation densities or currents alone difficult. 



The close-to-equilibrium regime Let us assume that the dynamics of the process 
is close to equilibrium, meaning that the transition rates arc of the form 

k(x,y) = k (x,y)e^ w ^ 

with ko(x, y) the equilibrium reference process. In such a dynamics, the stationary 
current j p = 0(e). In (|11.13p we can therefore drop the terms containing the 
stationary current to obtain 

?2^2 . -2 



x,y P 

which gives us the same conclusions as for the case of over damped diffusions: the 
current and occupation fluctuations decouple in this close-to-equilibrium regime. 
As a consequence a minimum entropy production principle can be derived for the 
occupation fluctuations and a maximum entropy production principle for currents. 

A question arises here especially for the occupation fluctuations, which was not 
visible for over damped diffusions: why does entropy production govern the close- 
to-equilibrium regime? The occupation fluctuation functional is expressed as an 
excess traffic. For over damped diffusions this was equal to an excess entropy 
production, but here it is not. 

However, in the closc-to-cquilibrium regime, the excess traffic and excess entropy 
production merge. We see this by inserting [i = po[l + e/ii] and k{x,y) = 
k (x,y)e^~ w ( x ' y ' > into the definitions of traffic and entropy production. Note that 
Po is the equilibrium distribution for the rates kg. Using subscripts W to denote 
that we work in the dynamics with W we get 

e 2 r 1 2 

o-w(M) = -j^2po(x) k o(x,y) m(x) ~ m(y) + W(x,y) +o(e 2 ) 
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- Hi(y)]W(x,y) + jW(x, y) 2 ] + o(e 2 ) 



Up to second order in e, the excess entropy production and traffic with respect to 
the equilibrium dynamics thus give the same: 



The reason that entropy production principles appear close to equilibrium is thus 
a result of two facts: The first is that the occupation and current fluctuations 
decouple, so that the joint fluctuation functional becomes a sum of the functionals 
for occupations and currents alone. Second, the traffic and entropy production 
become indistinguishable, so that we do not need traffic do describe these physical 
results. 



a w (v) - o-o(m) = nvO) ~ t o(m) + o(e 2 ) 



(11.15) 



Chapter 12 

Conclusions 



The questions and the methods covered in this part are not entirely original. They 
have appeared in the mathematical literature in a systematic way since the theory 
of large deviations was introduced in the framework of Markov processes, see the 
references in Chapter [5] and especially in Section 19.61 The relevance to physics 
and to statistical mechanics in particular is obvious, but the thermodynamic 
interpretation of the resulting dynamical fluctuation functionals has not been 
systematically investigated. A first study can be found in [55]. The research 
discussed in this part and in [7DE32 has added to that. There have of course been 
many other studies of dynamical fluctuation theory in the literature. 

We mention in particular the works of Derrida and Bodineau, see US] and of 
Bertini et al, e.g. in [5jE|. With respect to diffusion processes, our approach 
is especially similar to what one is doing for a macrostatistical theory where a 
hydrodynamic limit is taken by rcscaling of certain Markov jump processes. The 
hydrodynamic fluctuations can then be viewed as a solution of some infinite- 
dimensional diffusion process. Possible differences with the existing work are 
first of all that the problems related to the diffusion-approximation or to a 
hydrodynamic rescaling do not enter in our work. We just start from a finite- 
dimensional diffusion process or a Markov jump process as such, and without 
extra rescaling. In other words, we prefer to split the problem of hydrodynamical 
scaling with possible diffusion approximation from the problem of studying the 
dynamical fluctuations. 

The emphasis of our work is on the distinction between the time-symmetric and 
time-antisymmetric sector of the fluctuations. On the one hand the action of 
relative path-probability densities is split in entropy flux and traffic. On the 
other hand the observables are split into occupation density (time-symmetric) 
and current (time-antisymmetric). It then comes as no surprise that the joint 
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fluctuation functional /(/i, j) can be written as an excess traffic, and an excess 
entropy flux, which constitutes a first main conclusion, expressed in (|10.25|) and 
([ 1 1 . 1 L) p . The excess is an excess of the original process with respect to a modified 
process (by adding a force to the dynamics) in which the fluctuations are typical. 
Contractions to occupations or currents alone is hard to do. 

Our second main result covers occupation fluctuations, see ()10.34p and 
There one can see that the fluctuation functional can be expressed in terms of 
only an excess traffic, and the modified dynamics is made by adding a potential. 
Furthermore, we like to emphasize a structure in the joint fluctuations of density 
and current which reminds of thermodynamic potentials, which can be found in 
Sections HUH and EH 

Finally, for small fluctuations one can clearly see that the coupling between 
occupation and current fluctuations only vanishes when close to equilibrium. In 
that regime minimum and maximum entropy production principles emerge. 

In |72j we have investigated dynamical fluctuations for semi-Markov jump 
processes. These are an extension of Markov jump processes where the waiting 
times are not exponentially distributed. In |96| we extended the analysis of 
overdamped diffusion processes to time-dependent but periodic dynamics. In those 
cases the general conclusions we have outlined do remain true. 

However, it is clear that the story as presented in this part is far from finished. 
First of all, we have not provided immediately applicable results. We have rather 
begun to examine the general physical structure of dynamical fluctuations. To go 
on, other models should be studied to see which structures remain in more general 
settings. For example, one could see what happens for undcrdamped diffusions, as 
we did for fluctuation-dissipation relations. Large deviations in this setting have 
already been discussed in |10] , where a quantity similar to our traffic plays a major 
role. 



Part IV 

Conclusions and appendices 
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"While working on my note, I realized that I had met these guys (=traffic) before, 
when I had been trying to prove some of my conjectures on the driven lattice gas. 
Those conjectures came mainly from my 'intuition' that was unfortunately based 
on the equilibrium physics, and turned out to be wrong. Always it looked like 
the 'proof went OK, but at some stage I saw something went wrong because of 
unwanted factors coming from waiting times. That was traffic, and (in those days) 
I only regarded it as annoying noise. But your message probably is to make friends 
with them. I will try." 

Hal Tasaki, Department of Physics, Gakushuin University, 171-8588, JAPAN, in 

a recent e-mail conversation (September 2009) 



Chapter 13 

Overall conclusions 



This chapter is devoted to some general conclusions of this text. For conclusions 
about fluctuation-dissipation relations and dynamical fluctuations separately, we 
refer to Chavters[$\ and UM Basically we have used the same method throughout this 
text: on the level of stochastic trajectories we distinguish entropy and traffic, that 
completely specify the probabilities of those trajectories. Quantities and physical 
relations that are computed through averages over possible trajectories are thus 
governed by both entropy and traffic. Entropy is a known quantity: it has an 
operational definition. In this chapter we shortly discuss the status of the less- 
known traffic and its possible interpretations. 



13.1 Traffic for Markov jump processes and over- 
damped diffusions 

Wc have investigated both fluctuation dissipation-relations and dynamical fluctu- 
ations for Markov jump processes and overdamped diffusions. This warrants a 
comparison. Generally we have used the same strategy in both parts, namely 
using a description in terms of stochastic trajectories and more importantly the 
splitting of the action in a time-symmetric and time-antisymmetric part. 

The time-antisymmetric part of the action is, under the local detailed balance 
assumption, equal to the entropy flux into the environment. The restrictions 
under which this works is that the parts of the environment connected to the 
system, each arc and remain in equilibrium. This entropy flux can therefore be 
expressed in terms of the heat dissipation into that environment. 
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The time-symmetric part of the action is called traffic, ft is much more difficult to 
give this quantity a general physical meaning than for entropy flux. One reason for 
that, is that the concept only makes sense as an excess. This excess is a difference 
of the same quantity in two dynamics. Our results for Markov jump processes and 
overdamped diffusions do give us a tool to give more interpretation to traffic. 

13.1.1 Traffic as an escape rate 

We have seen that the rate function for the large deviations of the empirical 
occupation density /(/i) can be expressed as an excess in traffic (a traffic rate 
actually), see (|lU.34p and (|U.lip . 

How can we interpret this rate function? For this, first consider the set of 
distributions [i that vanish on a given subset A of the state space f2. The 
contraction principle then tells us that the infimum of I(j-i) over this set gives 
the probability of the particle not visiting this A. This infimum can thus be 
considered an escape rate from the subset A. The rate function /(//), and thus the 
excess traffic can then be called a generalized escape rate from the distribution fi. 

Another way of seeing this is by reversing our viewpoint. The original viewpoint is 
that, given a distribution \i we look for a potential V that makes it typical. Then 
we compute the excess traffic of the original with respect to this new dynamics with 
the added potential. The reverse viewpoint is that we start from some dynamics 
with a stationary density \i. We then add to the dynamics a potential — V. The 
change in the instantaneous traffic rate then gives the rate function of fj, in that new 
dynamics, in which it is no longer stationary. The bigger the rate function (excess 
traffic) the faster \i relaxes to the new stationary distribution, thus 'escaping from 
fx! An interesting observation follows from this reverse viewpoint: when a potential 
is added to a stationary system, the positivity of the rate function tells us that the 
traffic should increase. In what follows, we interpret traffic as activity, meaning 
that adding a potential increases the activity of the system. 

Also on the level of trajectories we can see this. For Markov jump processes we 
wrote (|f f ,5|) , which is a time-integral of the escape rates of the states the system 
has visited. Moreover, overdamped diffusions can be recovered from Markov jump 
processes when space and time are rescaled in a specific way (see Appendix [C| . 
Therefore we still interpret the traffic in terms of 'excess escape rates' even on the 
level of paths for overdamped diffusions. 
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13.1.2 Traffic as activity 

The excess traffic appearing in the rate function for the large deviations of the 
empirical occupation density is, for Markov jump processes, equal to 

I(fx) = r( M ) - t v ( M ) = £ rtx)k{x, y) - tix)Hx, y)efrM- v W 

x,y x,y 

This is an excess in the expected number of jumps per unit of time, when in the 
distribution fi. In this sense, the traffic can be called an activity. Again, because 
overdamped diffusions arc recovered as a scaling limit of Markov jump processes, 
we can still see the excess traffic as an excess activity for overdamped diffusion 
processes. 

When the fluctuation is small, i.e. fi is close to the stationary distribution, the 
potential V that makes /i typical, is also small, if we write \x = p[l + e/xi] and 
V —> eV, we get: 

7( M )=r( M )-rv(/i) = -^*E,p{x)iJii(x)k{x,v)\y{v)-V{x)] (13.1) 

x,y 

X 

and similarly for overdamped diffusions. Indeed, we recognize here the version of 
traffic t(x) = f3LV(x) that appeared in the fluctuation-dissipation relations (|6.6p . 



13.1.3 Unpopular traffic 

When comparing the fluctuation-dissipation relation out of equilibrium, but for 
one heat bath (|5 . 14[) with the equilibrium one (|5.2p . we see that the equilibrium 
fluctuation-dissipation theorem can be rewritten: 

Rgvit, s) = p^- s (V(x s )Q(x t ))l = - (t(w, S )Q(x t ))l 

The first equality is the usual one. In our framework we see that it is the correlation 
between the observable and the excess entropy flux of the perturbed with respect 
to the unperturbed process. The last equality states that we can also consider the 
correlation with the excess traffic. In equilibrium these are the same! 

Similarly, for dynamical fluctuations we have seen that close to equilibrium excess 
entropy production and excess traffic are the same, see (|11.15p . This may 
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explain why traffic has not appeared much earlier in statistical physics. Around 
equilibrium one does not see it as an independent new quantity and all attention 
goes to its more popular brother entropy. This explains results as minimum and 
maximum entropy production principles, (see Sections 110.71 and lll.6|) and the 
McLennan formula (Section I2.10p . 

Celebrated recent results concerning a far-from-equilibrium regime, like the 
fluctuation-theorem and work relations (see Section I2.10J) , only make statements 
about the time-antisymmetric part of the dynamics. Because of that entropy is 
sufficient to describe the results, avoiding traffic completely. 

It seems however, that for a full description of noncquilibrium systems, the time- 
symmetric sector of the process described by the traffic is indispensable. This 
constitutes a major message of this text: traffic should not be avoided, it should 
be investigated. 

13.2 Outlook 

Fluctuation-dissipation relations In part II of this text we have outlined the 
general strategy to derive fluctuation-dissipation relations out of equilibrium, as 
well as given several explicit examples, ready to be tested experimentally. But of 
course the study is not finished. Apart from applying the strategy to other models, 
several related problems seem very interesting and can be investigated in the same 
framework. We suggest some examples: 

• The response of a system with respect to a perturbation other than a 
potential. This could be for example an added nonconservative force, or 
a change in the chemical potential of one of the particle reservoirs. This 
should be perfectly doable within the framework outline in this text. 

• Changes in temperature of the heat baths as perturbations. This is a very 
interesting topic for applications in meteorology but also in noncquilibrium 
calorimetry. However, such a perturbation can be tricky in our framework: 
for example for diffusion processes: if the diffusion coefficient (which 
depends usually on the temperature) is perturbed, then the path-probability 
densities of the perturbed and unperturbed process are no longer absolutely 
continuous with respect to each other. 

• A step towards thermodynamics through quasistatic processes: suppose that 
amplitude h t of the perturbation potential is not small, but changes very 
slowly. So slowly that the system is always very close to the stationary 
distribution of the dynamics at that moment. In such cases the response 
relations we have derived can be used to write expressions for the excess 
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heat, work and change of energy. The second law of thermodynamics for 
example can be investigated in such processes. 

• The use of our results in models of spin-glasses, where the notion of effective 
temperature often is useful. As our approach gives the response explicitly 
in terms of correlation functions for Markov jump processes, which often 
model spin-glasses, it could be used to develop the knowledge of effective 
temperatures. 

Dynamical fluctuations In part III we have discussed dynamical fluctuations 
of occupations and currents in overdamped diffusions and Markov jump processes. 
Although some quite general conclusions were made in terms of entropy and traffic, 
and this theory adds physical interpretation to traffic as escape rate and as a 
'thermodynamic' potential, the story is far from finished. Some suggestions for 
future work: 

• An investigation of the connection between dynamical and static fluctuations, 
how one theory can be derived from the other, can give a more systematic 
understanding of large deviations in noncquilibrium statistical mechanics. 

• Dynamical fluctuations in undcrdamped diffusion processes can give more 
insight into the concept of traffic in these models. As is clear from Markov 
jump processes and overdamped diffusions, dynamical fluctuation theory is 
of great help here. 

• It would be good to have some explicit results, i.e. some explicit rate 
functions, which moreover relate to experimental accessible predictions. 
Simulations could also help in visualizing the rate functions, at least for 
simple systems. 



Appendix A 

The generator for Markov 
processes 

A.l Different generators 

Backward generator When the system evolves according to a Markov dynamics, 
there exists an operator L acting on state functions /, such that 

!</(* t )>, = (£/(x t )>, V/ (A.l) 
This is a differential equation with the following solution 

(f(x t )) Un = f dx^{x){e tL /)(*) 



where the integral should be replaced by a sum when the configuration space is 
discrete. In words, e tL "pulls a function f back to the time of the initial measure." 
L is therefore often called the backward generator. 



Forward generator There also exists a forward generator U, defined in the 
following way: for any pair of state functions /, g 

I dxg(x)Lf(x) = f dxf(x)L^g(x) (A.2) 
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If wc now consider the definition of the time-evolved distribution p t : 
dx/j (x)(e tL f)(x) = (f(x t )) un = / dxn t {x)f{x) 



we see that pt = e tL p,Q. So pushes the distribution forward in time, and is 
therefore called the forward generator. The forward generator can be found from 
the Master equation for Markov jump processes or the Fokkcr-Planck equation for 
diffusions, because 

(A.3) 



Adjoint generator For time-independent dynamics, the adjoint generator L* is 
defined with the help of the stationary distribution p: for any pair of state functions 
f,9 

dxp(x)g(x)L*f(x) = J dxp(x)f(x)Lg(x) (A.4) 

It is very important to realize that this adjoint generator depends on the stationary 
distribution, and is therefore often not explicitly known in nonequilibrium systems. 
This adjoint generator is related to the time-reversed dynamics. 



To apply time-reversal, we must remember that there are possibly variables that 
change sign under time-reversal, i.e. velocities. To simplify the analysis however, 
we restrict ourselves here to the case that we have only time-symmetric variables 
(Markov jump processes and overdamped diffusions). 

The stationary distribution for the time-reversed dynamics (with path-probability 
density V{6lS)) is also p, and for any /, g: 



(g(xo)f(x t )) = I dV p (uj)g{xo)f(xt) 
dV p {eu)g{x t )f{x ) 



(g(xt)f(x )) [ 
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where the 6 denotes that the average is taken in the time-reversed dynamics. 
Writing the generator of the time-reversed dynamics as L , we get 



dxp(x)g(x)e tL f(x) = / dxp{x)f{x)e tL g{x) 



Because of the definition (|A.4[) , we sec that L e = L* . More generally one can 
derive that L e = ttL*tt in the case that the configuration of the system contains 
velocities. In other words, up to some reversal of signs of velocities, the adjoint 
generator is the generator of the time-reversed dynamics. This also means that an 
equilibrium dynamics must satisfy L* = ttLtt. 

Here we also sec that the equilibrium distribution can be defined as the distribution 
that satisfies 

dxp(x)f(x)Lg(x) = J dxp(x)g(x)7rLnf(x) Wf,g (A.5) 



A. 2 Correlation functions 

With the definitions of generators, one can write correlation functions in several 
different ways as follows, for s < t 



(ffaUxs))^ = I dxMx)e sL (ge {t - s)L f)(x)dx 

dxg(x)e {t - s)L f(x)e sL ^ (x)dx 
dxfi s (x)g(x)e^- s '> L f(x)dx 



= (f(xt~ s )g(x )) tIs 

One can see from this, that the time derivative of such a correlation function can 
be quite complicated. For example, for s < t: 
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|(/(x t ) 5 (x s )) Mo = (Lf{x t )g(x s )} M (A.6) 
However, we see that in stationary averages, this simplifies to 

^ (f{xt)g{x s )) p = - (Lf(x t )g(x s )) p = --^ (f(x t )g(x s )) p 

When the generator occurs in a stationary correlation function, one can replace it 
by the adjoint generator in the following case, for s < t: 

(Lf(x t )g(x s )) p = J dxp(x)g(x)e^ L Lf(x)dx 

dxp(x)g(x)Le^ s)L f(x)dx 

dxp(x)L*g(x)e {t - s)L f(x)dx 
= (f(xt)L*g(x s )) p 

where we used in the second equality that L commutes with e( <-s ) i . However, we 
can not apply a similar procedure for (f(xt)Lg(x s )) p , because L and L* do not 
commute in general. 



A. 3 Generators for jump processes and diffusions 

As additional information, we give here the generators for the main models used 
in this thesis. The procedure to find them is straightforward. We start from the 
evolution equation for the distribution p,t , which is the Master equation for Markov 
jump processes and the Fokker-Planck equation for diffusions, to find the forward 
generator iJ: 

dm r + 



After that we use (|A.2|) to find the backward generator, and (|A.4|) to find the 
adjoint generator. 
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A. 3.1 Markov jump processes 

The generators for a general Markov jump process with transition rates k(x,y) 



are: 



tffJ,(x) = ^2lfi(y)k(y,x) - fi(x)k(x,y)] 
v 

Lf(x) = J2 k (x,y)lf(y)-f(x)} 



L*f(x) 



(A.7) 
(A.8) 

(A.9) 



A. 3. 2 Overdamped diffusions in more dimensions 

The generators for a general overdamped Langcvin dynamics (|4. 1 3[) are: 

L*n = -V • [ X Fn - DVp] 
Lf = F x Vf + V-(DVf) 

L*.f = -F X V/ + V-(£>V/) + 2^DV/ 

P 

A. 3. 3 Underdamped diffusions 

The generators for underdamped diffusions of the form (|7.ip arc: 



(A.10) 
(A.ll) 

(A.12) 



Lg 

L*9 



E 

i 

E 

i 

E 



<9/i d 



dU 



~ Vi dx~ ~ dv~^ fl ~~~ "wOd + Ato- 



dx 



dvf 



dg 



dU 



dg , n d 2 g 



' dx 



dx 



dvi 



dvf 



dg ft dU dg 9 2 J , „ n 

Vi- (fi - TiT- ~ m ili V i) — + A 7^-2 + 2£) 



9ui dvf 



(A.13) 

(A.14) 

1 <9p dg 

p dvi dv t 
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A derivation of the 
Fokker-Planck equation 



Starting from a Langevin equation it is possible to derive the corresponding Fokker- 
Planck equation. We show it here for an over damped Langevin equation in one 
dimension (|4.1(J|) : 

dx t = xF(x t )dt + V2DdB t 

We do this by deriving the generator L of this process. For this, take an arbitrary 
state function g, which is smooth. Take a small number tel. We consider the 
average 

{g( x t+e))un= / dxnt{x) / dyProb(x t+e = x + y\x t = x)g{x + y) 



where, for e small enough we can write 

Prob{x t+t = x + y\x t = x) = — cxp j - j^[y - xF(x)e} 2 



with the normalization N — y/inDe. We make a Taylor-expansion of g around 
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dyProb(x t +e = x + y\x t = x)g(x + y) 

t 



In this way we have reduced the problem to calculating Gaussian integrals. We 
find 

dg 

dyProb(x t+e = x + y\x t = x)g(x + y) = g(x) + evF(x) — (x) 

+eD0( a :)+o(e) 

So that in the limit of e — > we get 



(Lgixt))^ = lim 



da d 2 a 
dx[i t (x)lxF(x)-^(x) + D^(x)} 

This gives us the backward generator L. The forward generator ]J then defines 
the Fokker-Planck equation. It is easily found by (|A.2|) . 



The generators and Fokker-Planck equations for other and more general diffusion 
equations can be found by the same reasoning. 



Appendix C 



From Markov jump processes 
to overdamped diffusions 



It is possible to derive overdamped diffusions as a certain limit of Markov jump 
processes. As an example, we consider the case of a single particle performing a 
Markov jump process on a ring with TV sites, labelled x = 0, e, 2e, . . . , Ne with 
Ne =1 = 0. The particle can hop every time only one site to the left or to the 
right, with rates k(x, x±e). A Markov process is fully determined by its generator. 
We therefore prove that the generator of this jump process converges in a certain 
scaling for e — > to the generator of a diffusion process. The generator of the 
Markov jump process on a state function g gives: 



We expand the transition rates in orders of e and rescale them as follows with 



k(x, x + e) = —\k (x) + ek 1 (x) + o(e)] 

for the rates for hopping to the right. This corresponds to a scaling of space (the 
space between two sites is 1 /e) and of time (the escape rates of the process are 




y— x±e 



- [k(x, x + e) + k(x,x~ e)] [g(x + e) + g(x - e) - 2g(x)] 
+ ^[k(x,x + e) - k(x,x- e)][g(x + e) - g(x - e)] 



e ->■ 0: 



1 
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multiplied by 1/e 2 so that the process goes faster and faster). 
Furthermore, the local detailed balance condition tells us that 

k S X, l + e \ = ex P {PlW(x, x + e) + U(x) - U(x + c)]} 
k(x + e, x) 

where W(x,x + e) is the work of the nonconservative forcing when the particle 
travels a distance e to the right. We write it therefore as W(x, x + e) = ef(x) + o(e). 
This allows us to express the rates for hopping to the left as: 

k(x + e,x) = k(x,x + e)cxp{~/3ef(x) + /3eU'(x) +o(e)} 

= ^ M x ) + #i(^) - Pk Q {x)f{x) + pk {x)U'{x)] + o(e)] 
For our calculation, we need k{x, x — e), which is thus equal to 

i [k (x) + e[h{x) - k' (x) - pko{x)f(x) + pk {x)U'{xj\ + o(e)] 

Substituting this into the generator, we get the following limit: 

lim L e g(x) = k (x)g"(x) + [/3k (x)f(x) - pk (x)U'{x) + k' (x)]g'(x) 

which exactly defines the generator for a diffusion process with the Langcvin 
equation 



dx t = X{x)[f{x t ) - U'{x t )]dt + D'(x t )dt + y/2D(x t )dB t 



with D = ko and \ = /3fco- We therefore see the forcing with D' naturally emerging. 
Moreover, the local detailed balance condition for Markov jump processes, has the 
relation \ = f3D as a consequence after the rescaling. 
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